A multi-source feedback tool for measuring a subset of Pediatrics Milestones

The APPD LEARN—NBME Pediatrics Milestones Assessment Group

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.

Original languageEnglish (US)
Pages (from-to)995-1002
Number of pages8
JournalMedical Teacher
Volume38
Issue number10
DOIs
StatePublished - Oct 2 2016
Externally publishedYes

Fingerprint

rating
nurse
resident
staff
evidence

ASJC Scopus subject areas

  • Education

Cite this

The APPD LEARN—NBME Pediatrics Milestones Assessment Group (2016). A multi-source feedback tool for measuring a subset of Pediatrics Milestones. Medical Teacher, 38(10), 995-1002. https://doi.org/10.3109/0142159X.2016.1147646

A multi-source feedback tool for measuring a subset of Pediatrics Milestones. / The APPD LEARN—NBME Pediatrics Milestones Assessment Group.

In: Medical Teacher, Vol. 38, No. 10, 02.10.2016, p. 995-1002.

Research output: Contribution to journalArticle

The APPD LEARN—NBME Pediatrics Milestones Assessment Group 2016, 'A multi-source feedback tool for measuring a subset of Pediatrics Milestones', Medical Teacher, vol. 38, no. 10, pp. 995-1002. https://doi.org/10.3109/0142159X.2016.1147646
The APPD LEARN—NBME Pediatrics Milestones Assessment Group. A multi-source feedback tool for measuring a subset of Pediatrics Milestones. Medical Teacher. 2016 Oct 2;38(10):995-1002. https://doi.org/10.3109/0142159X.2016.1147646
The APPD LEARN—NBME Pediatrics Milestones Assessment Group. / A multi-source feedback tool for measuring a subset of Pediatrics Milestones. In: Medical Teacher. 2016 ; Vol. 38, No. 10. pp. 995-1002.
@article{2f5c41e6b4734637a153a2a415aaa30e,
title = "A multi-source feedback tool for measuring a subset of Pediatrics Milestones",
abstract = "Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40{\%}), senior residents (34{\%}), nurses (22{\%}), and other staff (4{\%}). Variance in ratings was associated primarily with rater (32{\%}) and learner (22{\%}). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.",
author = "{The APPD LEARN—NBME Pediatrics Milestones Assessment Group} and Alan Schwartz and Margolis, {Melissa J.} and Sara Multerer and Haftel, {Hilary M.} and Schumacher, {Daniel J.} and Kristen Wright and Rosalind Moffitt and Hicks, {Patricia J.} and Jeanine Ronan and Rebecca Tenney-Soeiro and Dawn Young and Teri Turner and Melodie Allison and Linda Waggoner-Fountain and Lisa Morris and Grace Caputo and Sandra Barker and Vasu Bhavaraju and Ana Velazquez and {del Rey}, {Javier Gonzales} and Sue Poynter and Jackie Berry and Ann Guillot and Karen Leonard and Kimberly Boland and Olivia Mittel and Amanda Osta and Michelle Barnes and Emri Tas and Jen McDonnell and Cynthia Powell and Black, {Nicole Paradise} and Lilly Chang and Jill Leavens-Maurer and Ulka Kothari and Robert Lee and Susie Buchter and Meredith Carter and Robyn Blair and Sandra Arnold and Bugnitz, {Mark C.} and Ann Burke and Clyman, {Stephen G.} and Thomas Rebbecchi and Colleen Canavan and Christa Chaffinch and Margaret Richmond and Rose, {Kathleen M.} and Leta Rose and Yelena Spector",
year = "2016",
month = "10",
day = "2",
doi = "10.3109/0142159X.2016.1147646",
language = "English (US)",
volume = "38",
pages = "995--1002",
journal = "Medical Teacher",
issn = "0142-159X",
publisher = "Informa Healthcare",
number = "10",

}

TY - JOUR

T1 - A multi-source feedback tool for measuring a subset of Pediatrics Milestones

AU - The APPD LEARN—NBME Pediatrics Milestones Assessment Group

AU - Schwartz, Alan

AU - Margolis, Melissa J.

AU - Multerer, Sara

AU - Haftel, Hilary M.

AU - Schumacher, Daniel J.

AU - Wright, Kristen

AU - Moffitt, Rosalind

AU - Hicks, Patricia J.

AU - Ronan, Jeanine

AU - Tenney-Soeiro, Rebecca

AU - Young, Dawn

AU - Turner, Teri

AU - Allison, Melodie

AU - Waggoner-Fountain, Linda

AU - Morris, Lisa

AU - Caputo, Grace

AU - Barker, Sandra

AU - Bhavaraju, Vasu

AU - Velazquez, Ana

AU - del Rey, Javier Gonzales

AU - Poynter, Sue

AU - Berry, Jackie

AU - Guillot, Ann

AU - Leonard, Karen

AU - Boland, Kimberly

AU - Mittel, Olivia

AU - Osta, Amanda

AU - Barnes, Michelle

AU - Tas, Emri

AU - McDonnell, Jen

AU - Powell, Cynthia

AU - Black, Nicole Paradise

AU - Chang, Lilly

AU - Leavens-Maurer, Jill

AU - Kothari, Ulka

AU - Lee, Robert

AU - Buchter, Susie

AU - Carter, Meredith

AU - Blair, Robyn

AU - Arnold, Sandra

AU - Bugnitz, Mark C.

AU - Burke, Ann

AU - Clyman, Stephen G.

AU - Rebbecchi, Thomas

AU - Canavan, Colleen

AU - Chaffinch, Christa

AU - Richmond, Margaret

AU - Rose, Kathleen M.

AU - Rose, Leta

AU - Spector, Yelena

PY - 2016/10/2

Y1 - 2016/10/2

N2 - Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.

AB - Background: The Pediatrics Milestones Assessment Pilot employed a new multisource feedback (MSF) instrument to assess nine Pediatrics Milestones among interns and subinterns in the inpatient context. Objective: To report validity evidence for the MSF tool for informing milestone classification decisions. Methods: We obtained MSF instruments by different raters per learner per rotation. We present evidence for validity based on the unified validity framework. Results: One hundred and ninety two interns and 41 subinterns at 18 Pediatrics residency programs received a total of 1084 MSF forms from faculty (40%), senior residents (34%), nurses (22%), and other staff (4%). Variance in ratings was associated primarily with rater (32%) and learner (22%). The milestone factor structure fit data better than simpler structures. In domains except professionalism, ratings by nurses were significantly lower than those by faculty and ratings by other staff were significantly higher. Ratings were higher when the rater observed the learner for longer periods and had a positive global opinion of the learner. Ratings of interns and subinterns did not differ, except for ratings by senior residents. MSF-based scales correlated with summative milestone scores. Conclusion: We obtain moderately reliable MSF ratings of interns and subinterns in the inpatient context to inform some milestone assignments.

UR - http://www.scopus.com/inward/record.url?scp=84961909209&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961909209&partnerID=8YFLogxK

U2 - 10.3109/0142159X.2016.1147646

DO - 10.3109/0142159X.2016.1147646

M3 - Article

C2 - 27027428

AN - SCOPUS:84961909209

VL - 38

SP - 995

EP - 1002

JO - Medical Teacher

JF - Medical Teacher

SN - 0142-159X

IS - 10

ER -