LapMentor metrics possess limited construct validity

Pamela B. Andreatta, Derek T. Woodrum, Paul G. Gauger, Rebecca M. Minter

Research output: Contribution to journalArticle

20 Scopus citations

Abstract

BACKGROUND: Many surgical training programs are introducing virtual-reality laparoscopic simulators into their curriculum. If a surgical simulator will be used to determine when a trainee has reached an "expert" level of performance, its evaluation metrics must accurately reflect varying levels of skill. The ability of a metric to differentiate novice from expert performance is referred to as construct validity. The present study was undertaken to determine whether the LapMentor's metrics demonstrate construct validity. METHODS: Medical students, residents and faculty laparoscopic surgeons (n = 5-14 per group) performed 5 consecutive repetitions of 6 laparoscopic skills tasks: 30°Camera Manipulation, Eye-Hand Coordination, Clipping/Grasping, Cutting, Electrocautery, and Translocation of Objects. The LapMentor measured performance in 4 to 12 parameters per task. Mean performance for each parameter was compared between subject groups for the first and fifth repetitions. Pairwise comparisons among the 3 groups were made by post hoc t-tests with Bonferroni technique. Significance was set at P < 0.05. RESULTS: Of the 6 tasks evaluated, only the Eye-Hand Coordination task (3/12 parameters) and the Clipping and Grasping (1/7 parameters) had expert-level discrimination when performance was compared after completion of 1 repetition. Comparison of the fifth repetition performance (representing the plateau of the learning curves), demonstrated that the parameters Time and Score had expert level discrimination on the Eye-Hand Coordination task, and Time on the Cutting task. The remaining LapMentor tasks evaluated did not exhibit the ability to differentiate level of expertise based on the built-in metrics on either repetition 1 or 5. CONCLUSIONS: The majority of the LapMentor tasks' metrics were unable to differentiate between laparoscopic experts and less skilled subjects. Therefore, performance on those tasks may not accurately reflect a subject's true level of ability. Feedback to the manufacturer about these findings may encourage the development of evaluation parameters with greater sensitivity.

Original languageEnglish (US)
Pages (from-to)16-25
Number of pages10
JournalSimulation in Healthcare
Volume3
Issue number1
DOIs
StatePublished - Mar 1 2008

Keywords

  • Construct validity
  • LapMentor laparoscopic simulator
  • Laparoscopy
  • Simulation
  • Virtual reality

ASJC Scopus subject areas

  • Epidemiology
  • Medicine (miscellaneous)
  • Education
  • Modeling and Simulation

Fingerprint Dive into the research topics of 'LapMentor metrics possess limited construct validity'. Together they form a unique fingerprint.

  • Cite this

    Andreatta, P. B., Woodrum, D. T., Gauger, P. G., & Minter, R. M. (2008). LapMentor metrics possess limited construct validity. Simulation in Healthcare, 3(1), 16-25. https://doi.org/10.1097/SIH.0b013e31816366b9