Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones

Teri L. Turner, Vasudha L. Bhavaraju, Ulana A. Luciw-Dubas, Patricia J. Hicks, Sara Multerer, Amanda Osta, Jennifer McDonnell, Sue Poynter, Daniel J. Schumacher, Rebecca Tenney-Soeiro, Linda Waggoner-Fountain, Alan Schwartz

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Purpose To investigate evidence for validity of faculty members' pediatric milestone (PM) ratings of interns (first-year residents) and subinterns (fourth-year medical students) on nine subcompetencies related to readiness to serve as a pediatric intern in the inpatient setting. Method The Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network (APPD LEARN) and the National Board of Medical Examiners collaborated to investigate the utility of assessments of the PMs for trainees' performance. Data from 32 subinterns and 179 interns at 17 programs were collected from July 2012 through April 2013. Observers used several tools to assess learners. At each site, a faculty member used these data to make judgments about the learner's current developmental milestone in each subcompetency. Linear mixed models were fitted to milestone judgments to examine their relationship with learner's rank and subcompetency. Results On a 5-point developmental scale, mean milestone levels for interns ranged from 3.20 (for the subcompetency Work effectively as a member of a team) to 3.72 (Humanism) and for subinterns from 2.89 (Organize and prioritize care) to 3.61 (Professionalization). Mean milestone ratings were significantly higher for the Professionalism competency (3.59-3.72) for all trainees compared with Patient Care (2.89-3.24) and Personal and Professional Development (3.33-3.51). Mean intern ratings were significantly higher than mean subintern ratings for all nine subcompetencies except Professionalization, Humanism, and Trustworthiness. Conclusions The PMs had a coherent internal structure and could distinguish between differing levels of trainees, which supports their validation for documenting developmental progression of pediatric trainees.

Original languageEnglish (US)
Pages (from-to)809-819
Number of pages11
JournalAcademic Medicine
Volume92
Issue number6
DOIs
StatePublished - Jun 1 2017
Externally publishedYes

Fingerprint

trainee
rating
humanism
professionalization
evidence
medical examiner
trustworthiness
patient care
medical student
director
resident
performance

ASJC Scopus subject areas

  • Education

Cite this

Turner, T. L., Bhavaraju, V. L., Luciw-Dubas, U. A., Hicks, P. J., Multerer, S., Osta, A., ... Schwartz, A. (2017). Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones. Academic Medicine, 92(6), 809-819. https://doi.org/10.1097/ACM.0000000000001622

Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones. / Turner, Teri L.; Bhavaraju, Vasudha L.; Luciw-Dubas, Ulana A.; Hicks, Patricia J.; Multerer, Sara; Osta, Amanda; McDonnell, Jennifer; Poynter, Sue; Schumacher, Daniel J.; Tenney-Soeiro, Rebecca; Waggoner-Fountain, Linda; Schwartz, Alan.

In: Academic Medicine, Vol. 92, No. 6, 01.06.2017, p. 809-819.

Research output: Contribution to journalArticle

Turner, TL, Bhavaraju, VL, Luciw-Dubas, UA, Hicks, PJ, Multerer, S, Osta, A, McDonnell, J, Poynter, S, Schumacher, DJ, Tenney-Soeiro, R, Waggoner-Fountain, L & Schwartz, A 2017, 'Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones', Academic Medicine, vol. 92, no. 6, pp. 809-819. https://doi.org/10.1097/ACM.0000000000001622
Turner, Teri L. ; Bhavaraju, Vasudha L. ; Luciw-Dubas, Ulana A. ; Hicks, Patricia J. ; Multerer, Sara ; Osta, Amanda ; McDonnell, Jennifer ; Poynter, Sue ; Schumacher, Daniel J. ; Tenney-Soeiro, Rebecca ; Waggoner-Fountain, Linda ; Schwartz, Alan. / Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones. In: Academic Medicine. 2017 ; Vol. 92, No. 6. pp. 809-819.
@article{d4004db563834a18b876603510dc20d9,
title = "Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones",
abstract = "Purpose To investigate evidence for validity of faculty members' pediatric milestone (PM) ratings of interns (first-year residents) and subinterns (fourth-year medical students) on nine subcompetencies related to readiness to serve as a pediatric intern in the inpatient setting. Method The Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network (APPD LEARN) and the National Board of Medical Examiners collaborated to investigate the utility of assessments of the PMs for trainees' performance. Data from 32 subinterns and 179 interns at 17 programs were collected from July 2012 through April 2013. Observers used several tools to assess learners. At each site, a faculty member used these data to make judgments about the learner's current developmental milestone in each subcompetency. Linear mixed models were fitted to milestone judgments to examine their relationship with learner's rank and subcompetency. Results On a 5-point developmental scale, mean milestone levels for interns ranged from 3.20 (for the subcompetency Work effectively as a member of a team) to 3.72 (Humanism) and for subinterns from 2.89 (Organize and prioritize care) to 3.61 (Professionalization). Mean milestone ratings were significantly higher for the Professionalism competency (3.59-3.72) for all trainees compared with Patient Care (2.89-3.24) and Personal and Professional Development (3.33-3.51). Mean intern ratings were significantly higher than mean subintern ratings for all nine subcompetencies except Professionalization, Humanism, and Trustworthiness. Conclusions The PMs had a coherent internal structure and could distinguish between differing levels of trainees, which supports their validation for documenting developmental progression of pediatric trainees.",
author = "Turner, {Teri L.} and Bhavaraju, {Vasudha L.} and Luciw-Dubas, {Ulana A.} and Hicks, {Patricia J.} and Sara Multerer and Amanda Osta and Jennifer McDonnell and Sue Poynter and Schumacher, {Daniel J.} and Rebecca Tenney-Soeiro and Linda Waggoner-Fountain and Alan Schwartz",
year = "2017",
month = "6",
day = "1",
doi = "10.1097/ACM.0000000000001622",
language = "English (US)",
volume = "92",
pages = "809--819",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "6",

}

TY - JOUR

T1 - Validity Evidence from Ratings of Pediatric Interns and Subinterns on a Subset of Pediatric Milestones

AU - Turner, Teri L.

AU - Bhavaraju, Vasudha L.

AU - Luciw-Dubas, Ulana A.

AU - Hicks, Patricia J.

AU - Multerer, Sara

AU - Osta, Amanda

AU - McDonnell, Jennifer

AU - Poynter, Sue

AU - Schumacher, Daniel J.

AU - Tenney-Soeiro, Rebecca

AU - Waggoner-Fountain, Linda

AU - Schwartz, Alan

PY - 2017/6/1

Y1 - 2017/6/1

N2 - Purpose To investigate evidence for validity of faculty members' pediatric milestone (PM) ratings of interns (first-year residents) and subinterns (fourth-year medical students) on nine subcompetencies related to readiness to serve as a pediatric intern in the inpatient setting. Method The Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network (APPD LEARN) and the National Board of Medical Examiners collaborated to investigate the utility of assessments of the PMs for trainees' performance. Data from 32 subinterns and 179 interns at 17 programs were collected from July 2012 through April 2013. Observers used several tools to assess learners. At each site, a faculty member used these data to make judgments about the learner's current developmental milestone in each subcompetency. Linear mixed models were fitted to milestone judgments to examine their relationship with learner's rank and subcompetency. Results On a 5-point developmental scale, mean milestone levels for interns ranged from 3.20 (for the subcompetency Work effectively as a member of a team) to 3.72 (Humanism) and for subinterns from 2.89 (Organize and prioritize care) to 3.61 (Professionalization). Mean milestone ratings were significantly higher for the Professionalism competency (3.59-3.72) for all trainees compared with Patient Care (2.89-3.24) and Personal and Professional Development (3.33-3.51). Mean intern ratings were significantly higher than mean subintern ratings for all nine subcompetencies except Professionalization, Humanism, and Trustworthiness. Conclusions The PMs had a coherent internal structure and could distinguish between differing levels of trainees, which supports their validation for documenting developmental progression of pediatric trainees.

AB - Purpose To investigate evidence for validity of faculty members' pediatric milestone (PM) ratings of interns (first-year residents) and subinterns (fourth-year medical students) on nine subcompetencies related to readiness to serve as a pediatric intern in the inpatient setting. Method The Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network (APPD LEARN) and the National Board of Medical Examiners collaborated to investigate the utility of assessments of the PMs for trainees' performance. Data from 32 subinterns and 179 interns at 17 programs were collected from July 2012 through April 2013. Observers used several tools to assess learners. At each site, a faculty member used these data to make judgments about the learner's current developmental milestone in each subcompetency. Linear mixed models were fitted to milestone judgments to examine their relationship with learner's rank and subcompetency. Results On a 5-point developmental scale, mean milestone levels for interns ranged from 3.20 (for the subcompetency Work effectively as a member of a team) to 3.72 (Humanism) and for subinterns from 2.89 (Organize and prioritize care) to 3.61 (Professionalization). Mean milestone ratings were significantly higher for the Professionalism competency (3.59-3.72) for all trainees compared with Patient Care (2.89-3.24) and Personal and Professional Development (3.33-3.51). Mean intern ratings were significantly higher than mean subintern ratings for all nine subcompetencies except Professionalization, Humanism, and Trustworthiness. Conclusions The PMs had a coherent internal structure and could distinguish between differing levels of trainees, which supports their validation for documenting developmental progression of pediatric trainees.

UR - http://www.scopus.com/inward/record.url?scp=85013421981&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85013421981&partnerID=8YFLogxK

U2 - 10.1097/ACM.0000000000001622

DO - 10.1097/ACM.0000000000001622

M3 - Article

C2 - 28557947

AN - SCOPUS:85013421981

VL - 92

SP - 809

EP - 819

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 6

ER -