Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients

Geoffrey M. Fleming, Richard B. Mink, Christoph Hornik, Amanda R. Emke, Michael L. Green, Katherine Mason, Toni Petrillo, Jennifer Schuette, M. Hossein Tcharmtchi, Margaret Winkler, David A. Turner

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.

RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).

CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.

BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.

OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.

Original languageEnglish (US)
Pages (from-to)346-352
Number of pages7
JournalJournal of graduate medical education
Volume8
Issue number3
DOIs
StatePublished - Jul 1 2016

Fingerprint

Central Venous Catheters
Critical Care
Observation
Pediatrics
Medicine
Central Venous Catheterization
Delphi Technique
Postal Service
ROC Curve
Mental Competency
Physicians
Surveys and Questionnaires

ASJC Scopus subject areas

  • Medicine(all)

Cite this

Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients. / Fleming, Geoffrey M.; Mink, Richard B.; Hornik, Christoph; Emke, Amanda R.; Green, Michael L.; Mason, Katherine; Petrillo, Toni; Schuette, Jennifer; Tcharmtchi, M. Hossein; Winkler, Margaret; Turner, David A.

In: Journal of graduate medical education, Vol. 8, No. 3, 01.07.2016, p. 346-352.

Research output: Contribution to journalArticle

Fleming, GM, Mink, RB, Hornik, C, Emke, AR, Green, ML, Mason, K, Petrillo, T, Schuette, J, Tcharmtchi, MH, Winkler, M & Turner, DA 2016, 'Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients', Journal of graduate medical education, vol. 8, no. 3, pp. 346-352. https://doi.org/10.4300/JGME-D-15-00365.1
Fleming, Geoffrey M. ; Mink, Richard B. ; Hornik, Christoph ; Emke, Amanda R. ; Green, Michael L. ; Mason, Katherine ; Petrillo, Toni ; Schuette, Jennifer ; Tcharmtchi, M. Hossein ; Winkler, Margaret ; Turner, David A. / Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients. In: Journal of graduate medical education. 2016 ; Vol. 8, No. 3. pp. 346-352.
@article{271c158dbc154a069b0380382e2404a5,
title = "Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients",
abstract = "METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3{\%} of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8{\%}) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8{\%} of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95{\%} CI 0.94-0.99).CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.",
author = "Fleming, {Geoffrey M.} and Mink, {Richard B.} and Christoph Hornik and Emke, {Amanda R.} and Green, {Michael L.} and Katherine Mason and Toni Petrillo and Jennifer Schuette and Tcharmtchi, {M. Hossein} and Margaret Winkler and Turner, {David A.}",
year = "2016",
month = "7",
day = "1",
doi = "10.4300/JGME-D-15-00365.1",
language = "English (US)",
volume = "8",
pages = "346--352",
journal = "Journal of graduate medical education",
issn = "1949-8349",
publisher = "University of Finance and Management",
number = "3",

}

TY - JOUR

T1 - Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients

AU - Fleming, Geoffrey M.

AU - Mink, Richard B.

AU - Hornik, Christoph

AU - Emke, Amanda R.

AU - Green, Michael L.

AU - Mason, Katherine

AU - Petrillo, Toni

AU - Schuette, Jennifer

AU - Tcharmtchi, M. Hossein

AU - Winkler, Margaret

AU - Turner, David A.

PY - 2016/7/1

Y1 - 2016/7/1

N2 - METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.

AB - METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.

UR - http://www.scopus.com/inward/record.url?scp=85015596558&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015596558&partnerID=8YFLogxK

U2 - 10.4300/JGME-D-15-00365.1

DO - 10.4300/JGME-D-15-00365.1

M3 - Article

C2 - 27413436

AN - SCOPUS:85015596558

VL - 8

SP - 346

EP - 352

JO - Journal of graduate medical education

JF - Journal of graduate medical education

SN - 1949-8349

IS - 3

ER -