Inter-Rater Reliability of the CASCADE Criteria: Challenges in Classifying Arteriopathies

Timothy J. Bernard, Lauren A. Beslow, Marilyn J. Manco-Johnson, Jennifer Armstrong-Wells, Richard Boada, David Weitzenkamp, Amanda Hollatz, Sharon Poisson, Catherine Amlie-Lefond, Warren Lo, Gabrielle DeVeber, Neil A. Goldenberg, Michael M. Dowling, E. Steve Roach, Heather J. Fullerton, Susanne M. Benseler, Lori C. Jordan, Adam Kirton, Rebecca N. Ichord

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Background and Purpose - There are limited data about the reliability of subtype classification in childhood arterial ischemic stroke, an issue that prompted the IPSS (International Pediatric Stroke Study) to develop the CASCADE criteria (Childhood AIS Standardized Classification and Diagnostic Evaluation). Our purpose was to determine the CASCADE criteria's reliability in a population of children with stroke. Methods - Eight raters from the IPSS reviewed neuroimaging and clinical records of 64 cases (16 cases each) randomly selected from a prospectively collected cohort of 113 children with arterial ischemic stroke and classified them using the CASCADE criteria. Clinical data abstracted included history of present illness, risk factors, and acute imaging. Agreement among raters was measured by unweighted κ statistic. Results - The CASCADE criteria demonstrated a moderate inter-rater reliability, with an overall κ statistic of 0.53 (95% confidence interval [CI]=0.39-0.67). Cardioembolic and bilateral cerebral arteriopathy subtypes had much higher agreement (κ=0.84; 95% CI=0.70-0.99; and κ=0.90; 95% CI=0.71-1.00, respectively) than cases of aortic/cervical arteriopathy (κ=0.36; 95% CI=0.01-0.71), unilateral focal cerebral arteriopathy of childhood (FCA; κ=0.49; 95% CI=0.23-0.76), and small vessel arteriopathy of childhood (κ=-0.012; 95% CI=-0.04 to 0.01). Conclusions - The CASCADE criteria have moderate reliability when used by trained and experienced raters, which suggests that it can be used for classification in multicenter pediatric stroke studies. However, the moderate reliability of the arteriopathic subtypes suggests that further refinement is needed for defining subtypes. Such revisions may reduce the variability in the literature describing risk factors, recurrence, and outcomes associated with childhood arteriopathy.

Original languageEnglish (US)
Pages (from-to)2443-2449
Number of pages7
JournalStroke
Volume47
Issue number10
DOIs
StatePublished - Oct 1 2016

Fingerprint

Stroke
Confidence Intervals
Pediatrics
Neuroimaging
Recurrence
Population

Keywords

  • attention
  • classification
  • cohort studies
  • consensus
  • incidence
  • pediatrics
  • risk factors

ASJC Scopus subject areas

  • Clinical Neurology
  • Cardiology and Cardiovascular Medicine
  • Advanced and Specialized Nursing

Cite this

Bernard, T. J., Beslow, L. A., Manco-Johnson, M. J., Armstrong-Wells, J., Boada, R., Weitzenkamp, D., ... Ichord, R. N. (2016). Inter-Rater Reliability of the CASCADE Criteria: Challenges in Classifying Arteriopathies. Stroke, 47(10), 2443-2449. https://doi.org/10.1161/STROKEAHA.116.013544

Inter-Rater Reliability of the CASCADE Criteria : Challenges in Classifying Arteriopathies. / Bernard, Timothy J.; Beslow, Lauren A.; Manco-Johnson, Marilyn J.; Armstrong-Wells, Jennifer; Boada, Richard; Weitzenkamp, David; Hollatz, Amanda; Poisson, Sharon; Amlie-Lefond, Catherine; Lo, Warren; DeVeber, Gabrielle; Goldenberg, Neil A.; Dowling, Michael M.; Roach, E. Steve; Fullerton, Heather J.; Benseler, Susanne M.; Jordan, Lori C.; Kirton, Adam; Ichord, Rebecca N.

In: Stroke, Vol. 47, No. 10, 01.10.2016, p. 2443-2449.

Research output: Contribution to journalArticle

Bernard, TJ, Beslow, LA, Manco-Johnson, MJ, Armstrong-Wells, J, Boada, R, Weitzenkamp, D, Hollatz, A, Poisson, S, Amlie-Lefond, C, Lo, W, DeVeber, G, Goldenberg, NA, Dowling, MM, Roach, ES, Fullerton, HJ, Benseler, SM, Jordan, LC, Kirton, A & Ichord, RN 2016, 'Inter-Rater Reliability of the CASCADE Criteria: Challenges in Classifying Arteriopathies', Stroke, vol. 47, no. 10, pp. 2443-2449. https://doi.org/10.1161/STROKEAHA.116.013544
Bernard TJ, Beslow LA, Manco-Johnson MJ, Armstrong-Wells J, Boada R, Weitzenkamp D et al. Inter-Rater Reliability of the CASCADE Criteria: Challenges in Classifying Arteriopathies. Stroke. 2016 Oct 1;47(10):2443-2449. https://doi.org/10.1161/STROKEAHA.116.013544
Bernard, Timothy J. ; Beslow, Lauren A. ; Manco-Johnson, Marilyn J. ; Armstrong-Wells, Jennifer ; Boada, Richard ; Weitzenkamp, David ; Hollatz, Amanda ; Poisson, Sharon ; Amlie-Lefond, Catherine ; Lo, Warren ; DeVeber, Gabrielle ; Goldenberg, Neil A. ; Dowling, Michael M. ; Roach, E. Steve ; Fullerton, Heather J. ; Benseler, Susanne M. ; Jordan, Lori C. ; Kirton, Adam ; Ichord, Rebecca N. / Inter-Rater Reliability of the CASCADE Criteria : Challenges in Classifying Arteriopathies. In: Stroke. 2016 ; Vol. 47, No. 10. pp. 2443-2449.
@article{2f44eb472ce6429b9ff58d3f14be0a54,
title = "Inter-Rater Reliability of the CASCADE Criteria: Challenges in Classifying Arteriopathies",
abstract = "Background and Purpose - There are limited data about the reliability of subtype classification in childhood arterial ischemic stroke, an issue that prompted the IPSS (International Pediatric Stroke Study) to develop the CASCADE criteria (Childhood AIS Standardized Classification and Diagnostic Evaluation). Our purpose was to determine the CASCADE criteria's reliability in a population of children with stroke. Methods - Eight raters from the IPSS reviewed neuroimaging and clinical records of 64 cases (16 cases each) randomly selected from a prospectively collected cohort of 113 children with arterial ischemic stroke and classified them using the CASCADE criteria. Clinical data abstracted included history of present illness, risk factors, and acute imaging. Agreement among raters was measured by unweighted κ statistic. Results - The CASCADE criteria demonstrated a moderate inter-rater reliability, with an overall κ statistic of 0.53 (95{\%} confidence interval [CI]=0.39-0.67). Cardioembolic and bilateral cerebral arteriopathy subtypes had much higher agreement (κ=0.84; 95{\%} CI=0.70-0.99; and κ=0.90; 95{\%} CI=0.71-1.00, respectively) than cases of aortic/cervical arteriopathy (κ=0.36; 95{\%} CI=0.01-0.71), unilateral focal cerebral arteriopathy of childhood (FCA; κ=0.49; 95{\%} CI=0.23-0.76), and small vessel arteriopathy of childhood (κ=-0.012; 95{\%} CI=-0.04 to 0.01). Conclusions - The CASCADE criteria have moderate reliability when used by trained and experienced raters, which suggests that it can be used for classification in multicenter pediatric stroke studies. However, the moderate reliability of the arteriopathic subtypes suggests that further refinement is needed for defining subtypes. Such revisions may reduce the variability in the literature describing risk factors, recurrence, and outcomes associated with childhood arteriopathy.",
keywords = "attention, classification, cohort studies, consensus, incidence, pediatrics, risk factors",
author = "Bernard, {Timothy J.} and Beslow, {Lauren A.} and Manco-Johnson, {Marilyn J.} and Jennifer Armstrong-Wells and Richard Boada and David Weitzenkamp and Amanda Hollatz and Sharon Poisson and Catherine Amlie-Lefond and Warren Lo and Gabrielle DeVeber and Goldenberg, {Neil A.} and Dowling, {Michael M.} and Roach, {E. Steve} and Fullerton, {Heather J.} and Benseler, {Susanne M.} and Jordan, {Lori C.} and Adam Kirton and Ichord, {Rebecca N.}",
year = "2016",
month = "10",
day = "1",
doi = "10.1161/STROKEAHA.116.013544",
language = "English (US)",
volume = "47",
pages = "2443--2449",
journal = "Stroke",
issn = "0039-2499",
publisher = "Lippincott Williams and Wilkins",
number = "10",

}

TY - JOUR

T1 - Inter-Rater Reliability of the CASCADE Criteria

T2 - Challenges in Classifying Arteriopathies

AU - Bernard, Timothy J.

AU - Beslow, Lauren A.

AU - Manco-Johnson, Marilyn J.

AU - Armstrong-Wells, Jennifer

AU - Boada, Richard

AU - Weitzenkamp, David

AU - Hollatz, Amanda

AU - Poisson, Sharon

AU - Amlie-Lefond, Catherine

AU - Lo, Warren

AU - DeVeber, Gabrielle

AU - Goldenberg, Neil A.

AU - Dowling, Michael M.

AU - Roach, E. Steve

AU - Fullerton, Heather J.

AU - Benseler, Susanne M.

AU - Jordan, Lori C.

AU - Kirton, Adam

AU - Ichord, Rebecca N.

PY - 2016/10/1

Y1 - 2016/10/1

N2 - Background and Purpose - There are limited data about the reliability of subtype classification in childhood arterial ischemic stroke, an issue that prompted the IPSS (International Pediatric Stroke Study) to develop the CASCADE criteria (Childhood AIS Standardized Classification and Diagnostic Evaluation). Our purpose was to determine the CASCADE criteria's reliability in a population of children with stroke. Methods - Eight raters from the IPSS reviewed neuroimaging and clinical records of 64 cases (16 cases each) randomly selected from a prospectively collected cohort of 113 children with arterial ischemic stroke and classified them using the CASCADE criteria. Clinical data abstracted included history of present illness, risk factors, and acute imaging. Agreement among raters was measured by unweighted κ statistic. Results - The CASCADE criteria demonstrated a moderate inter-rater reliability, with an overall κ statistic of 0.53 (95% confidence interval [CI]=0.39-0.67). Cardioembolic and bilateral cerebral arteriopathy subtypes had much higher agreement (κ=0.84; 95% CI=0.70-0.99; and κ=0.90; 95% CI=0.71-1.00, respectively) than cases of aortic/cervical arteriopathy (κ=0.36; 95% CI=0.01-0.71), unilateral focal cerebral arteriopathy of childhood (FCA; κ=0.49; 95% CI=0.23-0.76), and small vessel arteriopathy of childhood (κ=-0.012; 95% CI=-0.04 to 0.01). Conclusions - The CASCADE criteria have moderate reliability when used by trained and experienced raters, which suggests that it can be used for classification in multicenter pediatric stroke studies. However, the moderate reliability of the arteriopathic subtypes suggests that further refinement is needed for defining subtypes. Such revisions may reduce the variability in the literature describing risk factors, recurrence, and outcomes associated with childhood arteriopathy.

AB - Background and Purpose - There are limited data about the reliability of subtype classification in childhood arterial ischemic stroke, an issue that prompted the IPSS (International Pediatric Stroke Study) to develop the CASCADE criteria (Childhood AIS Standardized Classification and Diagnostic Evaluation). Our purpose was to determine the CASCADE criteria's reliability in a population of children with stroke. Methods - Eight raters from the IPSS reviewed neuroimaging and clinical records of 64 cases (16 cases each) randomly selected from a prospectively collected cohort of 113 children with arterial ischemic stroke and classified them using the CASCADE criteria. Clinical data abstracted included history of present illness, risk factors, and acute imaging. Agreement among raters was measured by unweighted κ statistic. Results - The CASCADE criteria demonstrated a moderate inter-rater reliability, with an overall κ statistic of 0.53 (95% confidence interval [CI]=0.39-0.67). Cardioembolic and bilateral cerebral arteriopathy subtypes had much higher agreement (κ=0.84; 95% CI=0.70-0.99; and κ=0.90; 95% CI=0.71-1.00, respectively) than cases of aortic/cervical arteriopathy (κ=0.36; 95% CI=0.01-0.71), unilateral focal cerebral arteriopathy of childhood (FCA; κ=0.49; 95% CI=0.23-0.76), and small vessel arteriopathy of childhood (κ=-0.012; 95% CI=-0.04 to 0.01). Conclusions - The CASCADE criteria have moderate reliability when used by trained and experienced raters, which suggests that it can be used for classification in multicenter pediatric stroke studies. However, the moderate reliability of the arteriopathic subtypes suggests that further refinement is needed for defining subtypes. Such revisions may reduce the variability in the literature describing risk factors, recurrence, and outcomes associated with childhood arteriopathy.

KW - attention

KW - classification

KW - cohort studies

KW - consensus

KW - incidence

KW - pediatrics

KW - risk factors

UR - http://www.scopus.com/inward/record.url?scp=84987899432&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84987899432&partnerID=8YFLogxK

U2 - 10.1161/STROKEAHA.116.013544

DO - 10.1161/STROKEAHA.116.013544

M3 - Article

C2 - 27633024

AN - SCOPUS:84987899432

VL - 47

SP - 2443

EP - 2449

JO - Stroke

JF - Stroke

SN - 0039-2499

IS - 10

ER -