Reported Pediatrics Milestones (Mostly) Measure Program, Not Learner Performance

Kimberly Hu, Patricia J. Hicks, Melissa Margolis, Carol Carraccio, Amanda Osta, Marcia L. Winward, Alan Schwartz

Research output: Contribution to journalArticlepeer-review

Abstract

PURPOSE: Semiannually, U.S. pediatrics residency programs report resident milestone levels to the Accreditation Council for Graduate Medical Education (ACGME). The Pediatrics Milestones Assessment Collaborative (PMAC, consisting of the National Board of Medical Examiners, American Board of Pediatrics, and Association of Pediatric Program Directors) developed workplace-based assessments of 2 inferences: readiness to serve as an intern with a supervisor present (D1) and readiness to care for patients with a supervisor nearby in the pediatric inpatient setting (D2). The authors compared learner and program variance in PMAC scores with ACGME milestones. METHOD: The authors examined sources of variance in PMAC scores and milestones between November 2015 and May 2017 of 181 interns at 8 U.S. pediatrics residency programs using random effects models with program, competency, learner, and program × competency components. RESULTS: Program-related milestone variance was substantial (54% D1, 68% D2), both in comparison to learner milestone variance (22% D1, 14% D2) and program variance in the PMAC scores (12% D1, 10% D2). In contrast, learner variance represented 44% (D1) or 26% (D2) of variance in PMAC scores. Within programs, PMAC scores were positively correlated with milestones for all but one competency. CONCLUSIONS: PMAC assessments provided scores with little program-specific variance and were more sensitive to differences in learners within programs compared with milestones. Milestones reflected greater differences by program than by learner. This may represent program-based differences in intern performance or in use of milestones as a reporting scale. Comparing individual learner milestones without adjusting for programs is problematic.

Original languageEnglish (US)
Pages (from-to)S89-S94
JournalAcademic medicine : journal of the Association of American Medical Colleges
Volume95
Issue number11S Association of American Medical Colleges Learn Serve Lead
DOIs
StatePublished - Nov 1 2020

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Reported Pediatrics Milestones (Mostly) Measure Program, Not Learner Performance'. Together they form a unique fingerprint.

Cite this