Meaningful Assessment of Robotic Surgical Style using the Wisdom of Crowds

M. Ershad, Robert V Rege, A. Majewicz Fey

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Objective: Quantitative assessment of surgical skills is an important aspect of surgical training; however, the proposed metrics are sometimes difficult to interpret and may not capture the stylistic characteristics that define expertise. This study proposes a methodology for evaluating the surgical skill, based on metrics associated with stylistic adjectives, and evaluates the ability of this method to differentiate expertise levels. Methods: We recruited subjects from different expertise levels to perform training tasks on a surgical simulator. A lexicon of contrasting adjective pairs, based on important skills for robotic surgery, inspired by the global evaluative assessment of robotic skills tool, was developed. To validate the use of stylistic adjectives for surgical skill assessment, posture videos of the subjects performing the task, as well as videos of the task were rated by crowd-workers. Metrics associated with each adjective were found using kinematic and physiological measurements through correlation with the crowd-sourced adjective assignment ratings. To evaluate the chosen metrics’ ability in distinguishing expertise levels, two classifiers were trained and tested using these metrics. Results: Crowd-assignment ratings for all adjectives were significantly correlated with expertise levels. The results indicate that naive Bayes classifier performs the best, with an accuracy of (Formula presented.), (Formula presented.), (Formula presented.), and (Formula presented.) when classifying into four, three, and two levels of expertise, respectively. Conclusion: The proposed method is effective at mapping understandable adjectives of expertise to the stylistic movements and physiological response of trainees.

Original languageEnglish (US)
Pages (from-to)1-12
Number of pages12
JournalInternational journal of computer assisted radiology and surgery
StateAccepted/In press - Mar 24 2018


  • Crowd-sourcing
  • Motion analysis
  • Robotic surgery
  • Surgical skill assessment

ASJC Scopus subject areas

  • Surgery
  • Radiology Nuclear Medicine and imaging
  • Health Informatics

Fingerprint Dive into the research topics of 'Meaningful Assessment of Robotic Surgical Style using the Wisdom of Crowds'. Together they form a unique fingerprint.

Cite this