Eye metrics as an objective assessment of surgical skill

Lee Richstone, Michael J. Schwartz, Casey Seideman, Jeffrey A Cadeddu, Sandra Marshall, Louis R. Kavoussi

Research output: Contribution to journalArticle

72 Citations (Scopus)

Abstract

Objective: Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill. Summary Background Data: Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997-999; Tanenhaus et al, Science. 1995;268:1632-1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663-668; Thomas and Lleras, Cognition. 2009;111:168-174; Schriver et al, Hum Factors. 2008;50:864-878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process. Methods: Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level. Results: Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9% and 92.9% accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0% and 90.7% accuracy, respectively. Conclusions: We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.

Original languageEnglish (US)
Pages (from-to)177-182
Number of pages6
JournalAnnals of Surgery
Volume252
Issue number1
DOIs
StatePublished - Jul 2010

Fingerprint

Eye Movements
Credentialing
Aptitude
Task Performance and Analysis
Operating Rooms
Cognition
Surgeons
Learning
Education

ASJC Scopus subject areas

  • Surgery

Cite this

Richstone, L., Schwartz, M. J., Seideman, C., Cadeddu, J. A., Marshall, S., & Kavoussi, L. R. (2010). Eye metrics as an objective assessment of surgical skill. Annals of Surgery, 252(1), 177-182. https://doi.org/10.1097/SLA.0b013e3181e464fb

Eye metrics as an objective assessment of surgical skill. / Richstone, Lee; Schwartz, Michael J.; Seideman, Casey; Cadeddu, Jeffrey A; Marshall, Sandra; Kavoussi, Louis R.

In: Annals of Surgery, Vol. 252, No. 1, 07.2010, p. 177-182.

Research output: Contribution to journalArticle

Richstone, L, Schwartz, MJ, Seideman, C, Cadeddu, JA, Marshall, S & Kavoussi, LR 2010, 'Eye metrics as an objective assessment of surgical skill', Annals of Surgery, vol. 252, no. 1, pp. 177-182. https://doi.org/10.1097/SLA.0b013e3181e464fb
Richstone L, Schwartz MJ, Seideman C, Cadeddu JA, Marshall S, Kavoussi LR. Eye metrics as an objective assessment of surgical skill. Annals of Surgery. 2010 Jul;252(1):177-182. https://doi.org/10.1097/SLA.0b013e3181e464fb
Richstone, Lee ; Schwartz, Michael J. ; Seideman, Casey ; Cadeddu, Jeffrey A ; Marshall, Sandra ; Kavoussi, Louis R. / Eye metrics as an objective assessment of surgical skill. In: Annals of Surgery. 2010 ; Vol. 252, No. 1. pp. 177-182.
@article{5381a30847a44bbaaf080f2dd3d05765,
title = "Eye metrics as an objective assessment of surgical skill",
abstract = "Objective: Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill. Summary Background Data: Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997-999; Tanenhaus et al, Science. 1995;268:1632-1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663-668; Thomas and Lleras, Cognition. 2009;111:168-174; Schriver et al, Hum Factors. 2008;50:864-878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process. Methods: Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level. Results: Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9{\%} and 92.9{\%} accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0{\%} and 90.7{\%} accuracy, respectively. Conclusions: We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.",
author = "Lee Richstone and Schwartz, {Michael J.} and Casey Seideman and Cadeddu, {Jeffrey A} and Sandra Marshall and Kavoussi, {Louis R.}",
year = "2010",
month = "7",
doi = "10.1097/SLA.0b013e3181e464fb",
language = "English (US)",
volume = "252",
pages = "177--182",
journal = "Annals of Surgery",
issn = "0003-4932",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Eye metrics as an objective assessment of surgical skill

AU - Richstone, Lee

AU - Schwartz, Michael J.

AU - Seideman, Casey

AU - Cadeddu, Jeffrey A

AU - Marshall, Sandra

AU - Kavoussi, Louis R.

PY - 2010/7

Y1 - 2010/7

N2 - Objective: Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill. Summary Background Data: Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997-999; Tanenhaus et al, Science. 1995;268:1632-1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663-668; Thomas and Lleras, Cognition. 2009;111:168-174; Schriver et al, Hum Factors. 2008;50:864-878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process. Methods: Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level. Results: Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9% and 92.9% accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0% and 90.7% accuracy, respectively. Conclusions: We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.

AB - Objective: Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill. Summary Background Data: Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997-999; Tanenhaus et al, Science. 1995;268:1632-1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663-668; Thomas and Lleras, Cognition. 2009;111:168-174; Schriver et al, Hum Factors. 2008;50:864-878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process. Methods: Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level. Results: Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9% and 92.9% accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0% and 90.7% accuracy, respectively. Conclusions: We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.

UR - http://www.scopus.com/inward/record.url?scp=77954089112&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77954089112&partnerID=8YFLogxK

U2 - 10.1097/SLA.0b013e3181e464fb

DO - 10.1097/SLA.0b013e3181e464fb

M3 - Article

C2 - 20562602

AN - SCOPUS:77954089112

VL - 252

SP - 177

EP - 182

JO - Annals of Surgery

JF - Annals of Surgery

SN - 0003-4932

IS - 1

ER -