Interrater reliability of clinical findings in children with possible appendicitis

Anupam B. Kharbanda, Michelle D. Stevenson, Charles G. Macias, Kelly Sinclair, Nanette C. Dudley, Jonathan Bennett, Lalit Bajaj, Manoj K. Mittal, Craig Huang, Richard G. Bachur, Peter S. Dayan

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

OBJECTIVE: Our objective was to determine the interrater reliability of clinical history and physical examination findings in children undergoing evaluation for possible appendicitis in a large, multicenter cohort. METHODS: We conducted a prospective, multicenter, cross-sectional study of children aged 3-18 years with possible appendicitis. Two clinicians independently evaluated patients and completed structured case report forms within 60 minutes of each other and without knowing the results of diagnostic imaging. We calculated raw agreement and assessed reliability by using the unweighted Cohen k statistic with 2-sided 95% confidence intervals. RESULTS: A total of 811 patients had 2 assessments completed, and 599 (74%) had 2 assessments completed within 60 minutes. Seventy-five percent of paired assessments were completed by pediatric emergency physicians. Raw agreement ranged from 64.9% to 92.3% for history variables and 4 of 6 variables had moderate interrater reliability (κ > .4). The highest k values were noted for duration of pain (κ = .56 [95% confidence intervals .51-.61]) and history of emesis (.84 [.80-.89]). For physical examination variables, raw agreement ranged from 60.9% to 98.7%, with 4 of 8 variables exhibiting moderate reliability. Among physical examination variables, the highest κ values were noted for abdominal pain with walking, jumping, or coughing (.54 [.45-.63]) and presence of any abdominal tenderness on examination (.49 [.19-.80]). CONCLUSIONS: Interrater reliability of patient history and physical examination variables was generally fair to moderate. Those variables with higher interrater reliability are more appropriate for inclusion in clinical prediction rules in children with possible appendicitis.

Original languageEnglish (US)
Pages (from-to)695-700
Number of pages6
JournalPediatrics
Volume129
Issue number4
DOIs
StatePublished - Apr 2012

Fingerprint

Appendicitis
Physical Examination
Confidence Intervals
Decision Support Techniques
Diagnostic Imaging
Abdominal Pain
Walking
Vomiting
Emergencies
Cross-Sectional Studies
History
Pediatrics
Physicians
Pain
Interrater Reliability
Physical

Keywords

  • Appendicitis
  • Clinical prediction rules
  • Interrater reliability

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Arts and Humanities (miscellaneous)

Cite this

Kharbanda, A. B., Stevenson, M. D., Macias, C. G., Sinclair, K., Dudley, N. C., Bennett, J., ... Dayan, P. S. (2012). Interrater reliability of clinical findings in children with possible appendicitis. Pediatrics, 129(4), 695-700. https://doi.org/10.1542/peds.2011-2037

Interrater reliability of clinical findings in children with possible appendicitis. / Kharbanda, Anupam B.; Stevenson, Michelle D.; Macias, Charles G.; Sinclair, Kelly; Dudley, Nanette C.; Bennett, Jonathan; Bajaj, Lalit; Mittal, Manoj K.; Huang, Craig; Bachur, Richard G.; Dayan, Peter S.

In: Pediatrics, Vol. 129, No. 4, 04.2012, p. 695-700.

Research output: Contribution to journalArticle

Kharbanda, AB, Stevenson, MD, Macias, CG, Sinclair, K, Dudley, NC, Bennett, J, Bajaj, L, Mittal, MK, Huang, C, Bachur, RG & Dayan, PS 2012, 'Interrater reliability of clinical findings in children with possible appendicitis', Pediatrics, vol. 129, no. 4, pp. 695-700. https://doi.org/10.1542/peds.2011-2037
Kharbanda AB, Stevenson MD, Macias CG, Sinclair K, Dudley NC, Bennett J et al. Interrater reliability of clinical findings in children with possible appendicitis. Pediatrics. 2012 Apr;129(4):695-700. https://doi.org/10.1542/peds.2011-2037
Kharbanda, Anupam B. ; Stevenson, Michelle D. ; Macias, Charles G. ; Sinclair, Kelly ; Dudley, Nanette C. ; Bennett, Jonathan ; Bajaj, Lalit ; Mittal, Manoj K. ; Huang, Craig ; Bachur, Richard G. ; Dayan, Peter S. / Interrater reliability of clinical findings in children with possible appendicitis. In: Pediatrics. 2012 ; Vol. 129, No. 4. pp. 695-700.
@article{5ff851361e8a4aed9d60e35ae2400661,
title = "Interrater reliability of clinical findings in children with possible appendicitis",
abstract = "OBJECTIVE: Our objective was to determine the interrater reliability of clinical history and physical examination findings in children undergoing evaluation for possible appendicitis in a large, multicenter cohort. METHODS: We conducted a prospective, multicenter, cross-sectional study of children aged 3-18 years with possible appendicitis. Two clinicians independently evaluated patients and completed structured case report forms within 60 minutes of each other and without knowing the results of diagnostic imaging. We calculated raw agreement and assessed reliability by using the unweighted Cohen k statistic with 2-sided 95{\%} confidence intervals. RESULTS: A total of 811 patients had 2 assessments completed, and 599 (74{\%}) had 2 assessments completed within 60 minutes. Seventy-five percent of paired assessments were completed by pediatric emergency physicians. Raw agreement ranged from 64.9{\%} to 92.3{\%} for history variables and 4 of 6 variables had moderate interrater reliability (κ > .4). The highest k values were noted for duration of pain (κ = .56 [95{\%} confidence intervals .51-.61]) and history of emesis (.84 [.80-.89]). For physical examination variables, raw agreement ranged from 60.9{\%} to 98.7{\%}, with 4 of 8 variables exhibiting moderate reliability. Among physical examination variables, the highest κ values were noted for abdominal pain with walking, jumping, or coughing (.54 [.45-.63]) and presence of any abdominal tenderness on examination (.49 [.19-.80]). CONCLUSIONS: Interrater reliability of patient history and physical examination variables was generally fair to moderate. Those variables with higher interrater reliability are more appropriate for inclusion in clinical prediction rules in children with possible appendicitis.",
keywords = "Appendicitis, Clinical prediction rules, Interrater reliability",
author = "Kharbanda, {Anupam B.} and Stevenson, {Michelle D.} and Macias, {Charles G.} and Kelly Sinclair and Dudley, {Nanette C.} and Jonathan Bennett and Lalit Bajaj and Mittal, {Manoj K.} and Craig Huang and Bachur, {Richard G.} and Dayan, {Peter S.}",
year = "2012",
month = "4",
doi = "10.1542/peds.2011-2037",
language = "English (US)",
volume = "129",
pages = "695--700",
journal = "Pediatrics",
issn = "0031-4005",
publisher = "American Academy of Pediatrics",
number = "4",

}

TY - JOUR

T1 - Interrater reliability of clinical findings in children with possible appendicitis

AU - Kharbanda, Anupam B.

AU - Stevenson, Michelle D.

AU - Macias, Charles G.

AU - Sinclair, Kelly

AU - Dudley, Nanette C.

AU - Bennett, Jonathan

AU - Bajaj, Lalit

AU - Mittal, Manoj K.

AU - Huang, Craig

AU - Bachur, Richard G.

AU - Dayan, Peter S.

PY - 2012/4

Y1 - 2012/4

N2 - OBJECTIVE: Our objective was to determine the interrater reliability of clinical history and physical examination findings in children undergoing evaluation for possible appendicitis in a large, multicenter cohort. METHODS: We conducted a prospective, multicenter, cross-sectional study of children aged 3-18 years with possible appendicitis. Two clinicians independently evaluated patients and completed structured case report forms within 60 minutes of each other and without knowing the results of diagnostic imaging. We calculated raw agreement and assessed reliability by using the unweighted Cohen k statistic with 2-sided 95% confidence intervals. RESULTS: A total of 811 patients had 2 assessments completed, and 599 (74%) had 2 assessments completed within 60 minutes. Seventy-five percent of paired assessments were completed by pediatric emergency physicians. Raw agreement ranged from 64.9% to 92.3% for history variables and 4 of 6 variables had moderate interrater reliability (κ > .4). The highest k values were noted for duration of pain (κ = .56 [95% confidence intervals .51-.61]) and history of emesis (.84 [.80-.89]). For physical examination variables, raw agreement ranged from 60.9% to 98.7%, with 4 of 8 variables exhibiting moderate reliability. Among physical examination variables, the highest κ values were noted for abdominal pain with walking, jumping, or coughing (.54 [.45-.63]) and presence of any abdominal tenderness on examination (.49 [.19-.80]). CONCLUSIONS: Interrater reliability of patient history and physical examination variables was generally fair to moderate. Those variables with higher interrater reliability are more appropriate for inclusion in clinical prediction rules in children with possible appendicitis.

AB - OBJECTIVE: Our objective was to determine the interrater reliability of clinical history and physical examination findings in children undergoing evaluation for possible appendicitis in a large, multicenter cohort. METHODS: We conducted a prospective, multicenter, cross-sectional study of children aged 3-18 years with possible appendicitis. Two clinicians independently evaluated patients and completed structured case report forms within 60 minutes of each other and without knowing the results of diagnostic imaging. We calculated raw agreement and assessed reliability by using the unweighted Cohen k statistic with 2-sided 95% confidence intervals. RESULTS: A total of 811 patients had 2 assessments completed, and 599 (74%) had 2 assessments completed within 60 minutes. Seventy-five percent of paired assessments were completed by pediatric emergency physicians. Raw agreement ranged from 64.9% to 92.3% for history variables and 4 of 6 variables had moderate interrater reliability (κ > .4). The highest k values were noted for duration of pain (κ = .56 [95% confidence intervals .51-.61]) and history of emesis (.84 [.80-.89]). For physical examination variables, raw agreement ranged from 60.9% to 98.7%, with 4 of 8 variables exhibiting moderate reliability. Among physical examination variables, the highest κ values were noted for abdominal pain with walking, jumping, or coughing (.54 [.45-.63]) and presence of any abdominal tenderness on examination (.49 [.19-.80]). CONCLUSIONS: Interrater reliability of patient history and physical examination variables was generally fair to moderate. Those variables with higher interrater reliability are more appropriate for inclusion in clinical prediction rules in children with possible appendicitis.

KW - Appendicitis

KW - Clinical prediction rules

KW - Interrater reliability

UR - http://www.scopus.com/inward/record.url?scp=84859408795&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859408795&partnerID=8YFLogxK

U2 - 10.1542/peds.2011-2037

DO - 10.1542/peds.2011-2037

M3 - Article

C2 - 22392173

AN - SCOPUS:84859408795

VL - 129

SP - 695

EP - 700

JO - Pediatrics

JF - Pediatrics

SN - 0031-4005

IS - 4

ER -