Interobserver variability among faculty in evaluations of residents' clinical skills

Joseph LaMantia, William Rennie, Donald A. Risucci, Rita Cydulka, Linda Spillane, Louis Graff, John Becher, Kurt Kleinschmidt

Research output: Contribution to journalArticle

13 Scopus citations

Abstract

Objective: To describe interobserver variability among emergency medicine (EM) faculty when using global assessment (GA) rating scales and performance-based criterion (PBC) checklists to evaluate EM residents' clinical skills during standardized patient (SP) encounters. Methods: Six EM residents were videotaped during encounters with SPs and subsequently evaluated by 38 EM faculty at four EM residency sites. There were two encounters in which a single SP presented with headache, two in which a second SP presented with chest pain, and two in which a third SP presented with abdominal pain, resulting in two parallel sets of three. Faculty used GA rating scales to evaluate history taking, physical examination, and interpersonal skills for the initial set of three cases. Each encounter in the second set was evaluated with complaint-specific PBC checklists developed by SAEM's National Consensus Group on Clinical Skills Task Force. Results: Standard deviations, computed for each score distribution, were generally similar across evaluation methods. None of the distributions deviated significantly from that of a Gaussian distribution, as indicated by the Kolmogorov-Smirnov goodness-of-fit test. On PBC checklists, 80% agreement among faculty observers was found for 74% of chest pain, 45% of headache, and 30% of abdominal pain items. Conclusions: When EM faculty evaluate clinical performance of EM residents during videotaped SP encounters, interobserver variabilities are similar, whether a PBC checklist or a GA rating scale is used.

Original languageEnglish (US)
Pages (from-to)38-44
Number of pages7
JournalAcademic Emergency Medicine
Volume6
Issue number1
DOIs
StatePublished - Jan 1999

Keywords

  • Clinical skills
  • Emergency medicine
  • Faculty
  • Interobserver variability
  • Performance assessment
  • Rating scales
  • Residents
  • Standardized patients

ASJC Scopus subject areas

  • Emergency Medicine

Fingerprint Dive into the research topics of 'Interobserver variability among faculty in evaluations of residents' clinical skills'. Together they form a unique fingerprint.

Cite this