TY - JOUR
T1 - Construct and face validity of MIST-VR, Endotower, and CELTS
T2 - Are we ready for skills assessment using simulators?
AU - Maithel, S.
AU - Sierra, R.
AU - Korndorffer, J.
AU - Neumann, P.
AU - Dawson, S.
AU - Callery, M.
AU - Jones, D.
AU - Scott, D.
PY - 2006/1/1
Y1 - 2006/1/1
N2 - Background: Video trainers may best offer visually realistic laparoscopic simulation, whereas virtual reality (VR) modules may best provide multidimensional objective measures of performance. This study compares the construct and face validity of three different laparoscopic simulators. Methods: Subjects were voluntarily enrolled at the Learning Center during the 2004 SAGES annual meeting. Each subject completed two repetitions of a single task on each of three simulators, MIST-VR, Endotower, and CELTS; performance scores were automatically generated and recorded. Scores of individuals with various levels of experience were compared to determine construct validity for each simulator. Experience was defined according to four parameters: (a) PGY level, (b) fellowship training, (c) basic laparoscopic cases, and (d) advanced laparoscopic cases. Subjects rated each simulator regarding six face validity (realism of simulation) parameters using a 10-point Likert scale (10 = best rating) and participant scores were compared to previously established expert scores (proficiency goals for training). Results: Ninety-one attendees completed the study. Construct validity was demonstrated for all three simulators; significant differences in scores were detected according to one parameter for MIST-VR, two param-eters for Endotower, and all four parameters for CELTS. Face validity was rated as good to excellent for all three simulators (7.0 ± 0.3 for MIST-VR, 7.9 ± 0.3 for Endotower [p < 0.001 vs MIST-VR], and 8.7 ± 0.1 for CELTS [p = 0.001 vs MIST-VR, p = 0.01 vs Endotower]); 6%, 0%, and 36% of "expert" participants obtained expert scores on MIST-VR, Endotower, and CELTS, respectively. Conclusions: All three simulators demonstrated significant construct and reasonable face validity. Although virtual reality holds great promise to expand the scope of laparoscopic simulation, current interfaces may limit their utility for assessment. Computer-enhanced video trainers may offer an improved interface while incorporating useful multidimensional metrics. Further work is needed to establish standards for appropriate skills assessment methods and performance levels using simulators.
AB - Background: Video trainers may best offer visually realistic laparoscopic simulation, whereas virtual reality (VR) modules may best provide multidimensional objective measures of performance. This study compares the construct and face validity of three different laparoscopic simulators. Methods: Subjects were voluntarily enrolled at the Learning Center during the 2004 SAGES annual meeting. Each subject completed two repetitions of a single task on each of three simulators, MIST-VR, Endotower, and CELTS; performance scores were automatically generated and recorded. Scores of individuals with various levels of experience were compared to determine construct validity for each simulator. Experience was defined according to four parameters: (a) PGY level, (b) fellowship training, (c) basic laparoscopic cases, and (d) advanced laparoscopic cases. Subjects rated each simulator regarding six face validity (realism of simulation) parameters using a 10-point Likert scale (10 = best rating) and participant scores were compared to previously established expert scores (proficiency goals for training). Results: Ninety-one attendees completed the study. Construct validity was demonstrated for all three simulators; significant differences in scores were detected according to one parameter for MIST-VR, two param-eters for Endotower, and all four parameters for CELTS. Face validity was rated as good to excellent for all three simulators (7.0 ± 0.3 for MIST-VR, 7.9 ± 0.3 for Endotower [p < 0.001 vs MIST-VR], and 8.7 ± 0.1 for CELTS [p = 0.001 vs MIST-VR, p = 0.01 vs Endotower]); 6%, 0%, and 36% of "expert" participants obtained expert scores on MIST-VR, Endotower, and CELTS, respectively. Conclusions: All three simulators demonstrated significant construct and reasonable face validity. Although virtual reality holds great promise to expand the scope of laparoscopic simulation, current interfaces may limit their utility for assessment. Computer-enhanced video trainers may offer an improved interface while incorporating useful multidimensional metrics. Further work is needed to establish standards for appropriate skills assessment methods and performance levels using simulators.
KW - CELTS
KW - Endotower
KW - Laparoscopy
KW - MIST-VR
KW - Simulator
KW - Validity
UR - http://www.scopus.com/inward/record.url?scp=30744477478&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=30744477478&partnerID=8YFLogxK
U2 - 10.1007/s00464-005-0054-4
DO - 10.1007/s00464-005-0054-4
M3 - Article
C2 - 16333535
AN - SCOPUS:30744477478
SN - 0930-2794
VL - 20
SP - 104
EP - 112
JO - Surgical Endoscopy and Other Interventional Techniques
JF - Surgical Endoscopy and Other Interventional Techniques
IS - 1
ER -