Inter- and intra-observer variability in the classification of extracapsular extension in p16 positive oropharyngeal squamous cell carcinoma nodal metastases

James S. Lewis, Yaman Tarabishy, Jingqin Luo, Haresh Mani, Justin A. Bishop, Marino E. Leon, Manju L. Prasad, Haodong Xu, Silvana Di Palma

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Summary Objective Discern inter- and intra-observer variability in the classification of extracapsular extension (ECE) in p16+ oropharyngeal (OP) SCC comparing pathologists' own criteria versus those of a well-defined classification system. Methods Five pathologists reviewed 50 digitally scanned nodal metastasis slides in three Rounds. Round One was by their own criteria as ECE present or absent, and Rounds Two and Three were with a defined ECE system: Grade 0 (no ECE), 0c (no ECE - thick capsule; no infiltration), 1 (ECE - cells beyond capsule), and 2 (soft tissue metastasis - cells in soft tissue without residual node). Round Three assessed intra-observer variability after an 8 month washout period. Results In Round One, all five agreed on only 48% of cases (n = 24). Fleiss's Kappa value was 0.508 (95% CI: 0.357-0.644). For Rounds Two and Three, Grades 0 and 0c and Grades 1 and 2 were separately grouped as ECE absent or present. In Round Two, all five agreed on 68% of cases (n = 34). Fleiss' Kappa was 0.635 (95% CI: 0.472-0.783), indicating statistically significantly better agreement. In Round Three, all five agreed on 64% of cases (n = 32) giving a Fleiss's Kappa of 0.639. Pathologists agreed with their prior reads in approximately 90% of cases (average n = 45.4, range n = 42-49), an average intra-observer Cohen's Kappa of 0.8 (range: 0.68-0.95). Inter- and intra-observer variability rates for classification of soft tissue metastasis (ECE2) were substantially worse. Conclusion There is substantial inter-, and modest intra-, observer variability among head and neck pathologists for ECE in p16+ OPSCC, which is modestly improved by a defined system.

Original languageEnglish (US)
Pages (from-to)985-990
Number of pages6
JournalOral Oncology
Volume51
Issue number11
DOIs
StatePublished - Nov 1 2015

Fingerprint

Observer Variation
Squamous Cell Carcinoma
Neoplasm Metastasis
Capsules
Neck
Head
Pathologists

Keywords

  • Extracapsular extension
  • Inter-observer variability
  • Lymph node metastasis
  • Oropharyngeal squamous cell carcinoma
  • p16
  • Soft tissue metastasis

ASJC Scopus subject areas

  • Oral Surgery
  • Oncology
  • Cancer Research

Cite this

Inter- and intra-observer variability in the classification of extracapsular extension in p16 positive oropharyngeal squamous cell carcinoma nodal metastases. / Lewis, James S.; Tarabishy, Yaman; Luo, Jingqin; Mani, Haresh; Bishop, Justin A.; Leon, Marino E.; Prasad, Manju L.; Xu, Haodong; Di Palma, Silvana.

In: Oral Oncology, Vol. 51, No. 11, 01.11.2015, p. 985-990.

Research output: Contribution to journalArticle

Lewis, James S. ; Tarabishy, Yaman ; Luo, Jingqin ; Mani, Haresh ; Bishop, Justin A. ; Leon, Marino E. ; Prasad, Manju L. ; Xu, Haodong ; Di Palma, Silvana. / Inter- and intra-observer variability in the classification of extracapsular extension in p16 positive oropharyngeal squamous cell carcinoma nodal metastases. In: Oral Oncology. 2015 ; Vol. 51, No. 11. pp. 985-990.
@article{14394c4c2f474e0a8c3d156570521a33,
title = "Inter- and intra-observer variability in the classification of extracapsular extension in p16 positive oropharyngeal squamous cell carcinoma nodal metastases",
abstract = "Summary Objective Discern inter- and intra-observer variability in the classification of extracapsular extension (ECE) in p16+ oropharyngeal (OP) SCC comparing pathologists' own criteria versus those of a well-defined classification system. Methods Five pathologists reviewed 50 digitally scanned nodal metastasis slides in three Rounds. Round One was by their own criteria as ECE present or absent, and Rounds Two and Three were with a defined ECE system: Grade 0 (no ECE), 0c (no ECE - thick capsule; no infiltration), 1 (ECE - cells beyond capsule), and 2 (soft tissue metastasis - cells in soft tissue without residual node). Round Three assessed intra-observer variability after an 8 month washout period. Results In Round One, all five agreed on only 48{\%} of cases (n = 24). Fleiss's Kappa value was 0.508 (95{\%} CI: 0.357-0.644). For Rounds Two and Three, Grades 0 and 0c and Grades 1 and 2 were separately grouped as ECE absent or present. In Round Two, all five agreed on 68{\%} of cases (n = 34). Fleiss' Kappa was 0.635 (95{\%} CI: 0.472-0.783), indicating statistically significantly better agreement. In Round Three, all five agreed on 64{\%} of cases (n = 32) giving a Fleiss's Kappa of 0.639. Pathologists agreed with their prior reads in approximately 90{\%} of cases (average n = 45.4, range n = 42-49), an average intra-observer Cohen's Kappa of 0.8 (range: 0.68-0.95). Inter- and intra-observer variability rates for classification of soft tissue metastasis (ECE2) were substantially worse. Conclusion There is substantial inter-, and modest intra-, observer variability among head and neck pathologists for ECE in p16+ OPSCC, which is modestly improved by a defined system.",
keywords = "Extracapsular extension, Inter-observer variability, Lymph node metastasis, Oropharyngeal squamous cell carcinoma, p16, Soft tissue metastasis",
author = "Lewis, {James S.} and Yaman Tarabishy and Jingqin Luo and Haresh Mani and Bishop, {Justin A.} and Leon, {Marino E.} and Prasad, {Manju L.} and Haodong Xu and {Di Palma}, Silvana",
year = "2015",
month = "11",
day = "1",
doi = "10.1016/j.oraloncology.2015.08.003",
language = "English (US)",
volume = "51",
pages = "985--990",
journal = "Oral Oncology",
issn = "1368-8375",
publisher = "Elsevier Limited",
number = "11",

}

TY - JOUR

T1 - Inter- and intra-observer variability in the classification of extracapsular extension in p16 positive oropharyngeal squamous cell carcinoma nodal metastases

AU - Lewis, James S.

AU - Tarabishy, Yaman

AU - Luo, Jingqin

AU - Mani, Haresh

AU - Bishop, Justin A.

AU - Leon, Marino E.

AU - Prasad, Manju L.

AU - Xu, Haodong

AU - Di Palma, Silvana

PY - 2015/11/1

Y1 - 2015/11/1

N2 - Summary Objective Discern inter- and intra-observer variability in the classification of extracapsular extension (ECE) in p16+ oropharyngeal (OP) SCC comparing pathologists' own criteria versus those of a well-defined classification system. Methods Five pathologists reviewed 50 digitally scanned nodal metastasis slides in three Rounds. Round One was by their own criteria as ECE present or absent, and Rounds Two and Three were with a defined ECE system: Grade 0 (no ECE), 0c (no ECE - thick capsule; no infiltration), 1 (ECE - cells beyond capsule), and 2 (soft tissue metastasis - cells in soft tissue without residual node). Round Three assessed intra-observer variability after an 8 month washout period. Results In Round One, all five agreed on only 48% of cases (n = 24). Fleiss's Kappa value was 0.508 (95% CI: 0.357-0.644). For Rounds Two and Three, Grades 0 and 0c and Grades 1 and 2 were separately grouped as ECE absent or present. In Round Two, all five agreed on 68% of cases (n = 34). Fleiss' Kappa was 0.635 (95% CI: 0.472-0.783), indicating statistically significantly better agreement. In Round Three, all five agreed on 64% of cases (n = 32) giving a Fleiss's Kappa of 0.639. Pathologists agreed with their prior reads in approximately 90% of cases (average n = 45.4, range n = 42-49), an average intra-observer Cohen's Kappa of 0.8 (range: 0.68-0.95). Inter- and intra-observer variability rates for classification of soft tissue metastasis (ECE2) were substantially worse. Conclusion There is substantial inter-, and modest intra-, observer variability among head and neck pathologists for ECE in p16+ OPSCC, which is modestly improved by a defined system.

AB - Summary Objective Discern inter- and intra-observer variability in the classification of extracapsular extension (ECE) in p16+ oropharyngeal (OP) SCC comparing pathologists' own criteria versus those of a well-defined classification system. Methods Five pathologists reviewed 50 digitally scanned nodal metastasis slides in three Rounds. Round One was by their own criteria as ECE present or absent, and Rounds Two and Three were with a defined ECE system: Grade 0 (no ECE), 0c (no ECE - thick capsule; no infiltration), 1 (ECE - cells beyond capsule), and 2 (soft tissue metastasis - cells in soft tissue without residual node). Round Three assessed intra-observer variability after an 8 month washout period. Results In Round One, all five agreed on only 48% of cases (n = 24). Fleiss's Kappa value was 0.508 (95% CI: 0.357-0.644). For Rounds Two and Three, Grades 0 and 0c and Grades 1 and 2 were separately grouped as ECE absent or present. In Round Two, all five agreed on 68% of cases (n = 34). Fleiss' Kappa was 0.635 (95% CI: 0.472-0.783), indicating statistically significantly better agreement. In Round Three, all five agreed on 64% of cases (n = 32) giving a Fleiss's Kappa of 0.639. Pathologists agreed with their prior reads in approximately 90% of cases (average n = 45.4, range n = 42-49), an average intra-observer Cohen's Kappa of 0.8 (range: 0.68-0.95). Inter- and intra-observer variability rates for classification of soft tissue metastasis (ECE2) were substantially worse. Conclusion There is substantial inter-, and modest intra-, observer variability among head and neck pathologists for ECE in p16+ OPSCC, which is modestly improved by a defined system.

KW - Extracapsular extension

KW - Inter-observer variability

KW - Lymph node metastasis

KW - Oropharyngeal squamous cell carcinoma

KW - p16

KW - Soft tissue metastasis

UR - http://www.scopus.com/inward/record.url?scp=84945454326&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84945454326&partnerID=8YFLogxK

U2 - 10.1016/j.oraloncology.2015.08.003

DO - 10.1016/j.oraloncology.2015.08.003

M3 - Article

C2 - 26293844

AN - SCOPUS:84945454326

VL - 51

SP - 985

EP - 990

JO - Oral Oncology

JF - Oral Oncology

SN - 1368-8375

IS - 11

ER -