Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation

Are they telling the same story?

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Background: The goal of this article is to examine the role of simulation in documenting resident nontechnical competencies outlined by the general surgery milestones, and compare those assessments to faculty ratings. Methods: Trainees completed a multiphasic simulation scenario to assess ability to obtain informed consent, lead a preoperative time out, crisis management, communication, and delivering bad news. Assessments from this scenario were compared with ratings of these same competencies collected from clinical rotations. Results: Twenty-six PGY1 trainees participated in the training program. Results revealed no significant correlations between simulation performance and faculty rotation ratings for any of the 5 competencies. With the exception of communication in the operating room, faculty reported an overall inability to reliably observe these competencies 26% to 41% of the time. Conclusions: This work suggests that traditional end-of-rotation evaluations may not be the most feasible and valid method to evaluate resident nontechnical skills for milestone assessment. Simulation-based assessments should be considered to fill this gap.

Original languageEnglish (US)
JournalAmerican Journal of Surgery
DOIs
StateAccepted/In press - 2016

Fingerprint

Communication
Aptitude
Clinical Competence
Operating Rooms
Informed Consent
Education

Keywords

  • Assessment
  • Evaluation
  • Milestones
  • Simulation

ASJC Scopus subject areas

  • Surgery

Cite this

@article{03b90eaea5314c8b856a5ef24d21d085,
title = "Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation: Are they telling the same story?",
abstract = "Background: The goal of this article is to examine the role of simulation in documenting resident nontechnical competencies outlined by the general surgery milestones, and compare those assessments to faculty ratings. Methods: Trainees completed a multiphasic simulation scenario to assess ability to obtain informed consent, lead a preoperative time out, crisis management, communication, and delivering bad news. Assessments from this scenario were compared with ratings of these same competencies collected from clinical rotations. Results: Twenty-six PGY1 trainees participated in the training program. Results revealed no significant correlations between simulation performance and faculty rotation ratings for any of the 5 competencies. With the exception of communication in the operating room, faculty reported an overall inability to reliably observe these competencies 26{\%} to 41{\%} of the time. Conclusions: This work suggests that traditional end-of-rotation evaluations may not be the most feasible and valid method to evaluate resident nontechnical skills for milestone assessment. Simulation-based assessments should be considered to fill this gap.",
keywords = "Assessment, Evaluation, Milestones, Simulation",
author = "Gardner, {Aimee K.} and Kareem AbdelFattah",
year = "2016",
doi = "10.1016/j.amjsurg.2016.07.025",
language = "English (US)",
journal = "American Journal of Surgery",
issn = "0002-9610",
publisher = "Elsevier Inc.",

}

TY - JOUR

T1 - Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation

T2 - Are they telling the same story?

AU - Gardner, Aimee K.

AU - AbdelFattah, Kareem

PY - 2016

Y1 - 2016

N2 - Background: The goal of this article is to examine the role of simulation in documenting resident nontechnical competencies outlined by the general surgery milestones, and compare those assessments to faculty ratings. Methods: Trainees completed a multiphasic simulation scenario to assess ability to obtain informed consent, lead a preoperative time out, crisis management, communication, and delivering bad news. Assessments from this scenario were compared with ratings of these same competencies collected from clinical rotations. Results: Twenty-six PGY1 trainees participated in the training program. Results revealed no significant correlations between simulation performance and faculty rotation ratings for any of the 5 competencies. With the exception of communication in the operating room, faculty reported an overall inability to reliably observe these competencies 26% to 41% of the time. Conclusions: This work suggests that traditional end-of-rotation evaluations may not be the most feasible and valid method to evaluate resident nontechnical skills for milestone assessment. Simulation-based assessments should be considered to fill this gap.

AB - Background: The goal of this article is to examine the role of simulation in documenting resident nontechnical competencies outlined by the general surgery milestones, and compare those assessments to faculty ratings. Methods: Trainees completed a multiphasic simulation scenario to assess ability to obtain informed consent, lead a preoperative time out, crisis management, communication, and delivering bad news. Assessments from this scenario were compared with ratings of these same competencies collected from clinical rotations. Results: Twenty-six PGY1 trainees participated in the training program. Results revealed no significant correlations between simulation performance and faculty rotation ratings for any of the 5 competencies. With the exception of communication in the operating room, faculty reported an overall inability to reliably observe these competencies 26% to 41% of the time. Conclusions: This work suggests that traditional end-of-rotation evaluations may not be the most feasible and valid method to evaluate resident nontechnical skills for milestone assessment. Simulation-based assessments should be considered to fill this gap.

KW - Assessment

KW - Evaluation

KW - Milestones

KW - Simulation

UR - http://www.scopus.com/inward/record.url?scp=85021303079&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021303079&partnerID=8YFLogxK

U2 - 10.1016/j.amjsurg.2016.07.025

DO - 10.1016/j.amjsurg.2016.07.025

M3 - Article

JO - American Journal of Surgery

JF - American Journal of Surgery

SN - 0002-9610

ER -