Surgical skill level assessment using automatic feature extraction methods

Objective and automatic evaluation of surgical skill is important for the design of surgical simulators used in surgical robotics training. Extensive research has been done to identify and evaluate a variety of evaluation metrics (e.g., path length, completion time); however, these metrics are only provided to the user after completion of the task, and may not fully use the underlying information in the movement data. This study proposes a method for automatic and objective evaluation of surgical expertise levels, in short time intervals, during task performance. We first compare three different automatic feature extraction methods including: (1) principle component analysis (PCA), (2) independent component analysis (ICA), and (3) linear discriminant analysis (LDA) on low-level position data, in their ability to distinguish among different expertise levels. We then study the performance of the best feature extraction method in different time intervals, for the purpose of finding the minimal time frame that accurately predicts user skill level. 14 subjects of different expertise levels were recruited to perform two simulated tasks on the da Vinci training simulator. The position of the subjects' arm joints (shoulder, elbow and wrist) in the dominant hand, as well as the position of both hands, were recorded. Four classifiers (Naive Bayes, support vector machine, nearest neighbor, and Decision Tree) were used to identify the best feature extraction method. The results indicate that PCA in combination with support vector machine can classify expertise levels with an accuracy of 98% in time frames of 0.25 seconds.