Agreement between cytotechnologists and cytopathologists as a new measure of cytopathologist performance in gynecologic cytology

Research output: Contribution to journalArticle

Abstract

BACKGROUND: Although objective measures of cytotechnologist (CT) and cytopathologist (CP) performance exist, challenges remain. Two assumptions deserve examination: CPs' interpretations are correct, and CTs and CPs render interpretations independently of each other. This study presents a CT-CP interpretation comparison and provides insight into these assumptions. METHODS: Every gynecologic cytology specimen examined by both a CT and a CP from December 2004 to March 2015 was extracted from the laboratory information system; glandular interpretations were excluded. Excel and SAS were used for CT-CP pair analysis. CT-CP pairs with fewer than 32 specimens (the lowest quartile) were excluded. For the remaining CT-CP pairs, 30 specimens or 10% of the specimens (whichever was higher) were randomly selected for comparison by a weighted κ statistic. κ values greater than 0.6 represented good agreement within CT-CP pairs. RESULTS: This study evaluated 7116 of 53,241 gynecologic cytology specimens (13.4%) that received CT and CP interpretations. This resulted in 155 pair-specific κ values from 15 CTs and 16 CPs. In aggregate, the κ values had a mean of 0.64, a standard deviation of 0.14, a median of 0.65, and a range of 0.27 to 0.91. Nine CTs exhibited good agreement in the majority of their pair-specific κ values with CPs (high-concordance CTs; 88 pair-specific κ values). This allowed us to identify outlier CPs who did not demonstrate good agreement with high-concordance CTs (16 of 88 pair-specific κ values [18.2%]). CONCLUSIONS: Laboratories can use this κ to determine when CP levels of agreement with CTs depart from those of their peers. Adding this to established metrics can give a more nuanced impression of CP performance.

Original languageEnglish (US)
JournalCancer cytopathology
DOIs
StateAccepted/In press - 2017

Fingerprint

Cell Biology
Clinical Laboratory Information Systems

Keywords

  • Agreement
  • Cytopathologist
  • Cytotechnologist
  • Gynecologic
  • Outlier
  • Papanicolaou test
  • Performance
  • Quality
  • SQL
  • κ

ASJC Scopus subject areas

  • Oncology
  • Cancer Research

Cite this

@article{6411aa6ca25f45a0ae155e6a0cdfc0dd,
title = "Agreement between cytotechnologists and cytopathologists as a new measure of cytopathologist performance in gynecologic cytology",
abstract = "BACKGROUND: Although objective measures of cytotechnologist (CT) and cytopathologist (CP) performance exist, challenges remain. Two assumptions deserve examination: CPs' interpretations are correct, and CTs and CPs render interpretations independently of each other. This study presents a CT-CP interpretation comparison and provides insight into these assumptions. METHODS: Every gynecologic cytology specimen examined by both a CT and a CP from December 2004 to March 2015 was extracted from the laboratory information system; glandular interpretations were excluded. Excel and SAS were used for CT-CP pair analysis. CT-CP pairs with fewer than 32 specimens (the lowest quartile) were excluded. For the remaining CT-CP pairs, 30 specimens or 10{\%} of the specimens (whichever was higher) were randomly selected for comparison by a weighted κ statistic. κ values greater than 0.6 represented good agreement within CT-CP pairs. RESULTS: This study evaluated 7116 of 53,241 gynecologic cytology specimens (13.4{\%}) that received CT and CP interpretations. This resulted in 155 pair-specific κ values from 15 CTs and 16 CPs. In aggregate, the κ values had a mean of 0.64, a standard deviation of 0.14, a median of 0.65, and a range of 0.27 to 0.91. Nine CTs exhibited good agreement in the majority of their pair-specific κ values with CPs (high-concordance CTs; 88 pair-specific κ values). This allowed us to identify outlier CPs who did not demonstrate good agreement with high-concordance CTs (16 of 88 pair-specific κ values [18.2{\%}]). CONCLUSIONS: Laboratories can use this κ to determine when CP levels of agreement with CTs depart from those of their peers. Adding this to established metrics can give a more nuanced impression of CP performance.",
keywords = "Agreement, Cytopathologist, Cytotechnologist, Gynecologic, Outlier, Papanicolaou test, Performance, Quality, SQL, κ",
author = "Quinn, {Andrew M.} and Minhajuddin, {Abu T.} and Hynan, {Linda S.} and Reisch, {Joan S.} and Cibas, {Edmund S.}",
year = "2017",
doi = "10.1002/cncy.21856",
language = "English (US)",
journal = "Cancer cytopathology",
issn = "1934-662X",
publisher = "Wiley-Blackwell",

}

TY - JOUR

T1 - Agreement between cytotechnologists and cytopathologists as a new measure of cytopathologist performance in gynecologic cytology

AU - Quinn, Andrew M.

AU - Minhajuddin, Abu T.

AU - Hynan, Linda S.

AU - Reisch, Joan S.

AU - Cibas, Edmund S.

PY - 2017

Y1 - 2017

N2 - BACKGROUND: Although objective measures of cytotechnologist (CT) and cytopathologist (CP) performance exist, challenges remain. Two assumptions deserve examination: CPs' interpretations are correct, and CTs and CPs render interpretations independently of each other. This study presents a CT-CP interpretation comparison and provides insight into these assumptions. METHODS: Every gynecologic cytology specimen examined by both a CT and a CP from December 2004 to March 2015 was extracted from the laboratory information system; glandular interpretations were excluded. Excel and SAS were used for CT-CP pair analysis. CT-CP pairs with fewer than 32 specimens (the lowest quartile) were excluded. For the remaining CT-CP pairs, 30 specimens or 10% of the specimens (whichever was higher) were randomly selected for comparison by a weighted κ statistic. κ values greater than 0.6 represented good agreement within CT-CP pairs. RESULTS: This study evaluated 7116 of 53,241 gynecologic cytology specimens (13.4%) that received CT and CP interpretations. This resulted in 155 pair-specific κ values from 15 CTs and 16 CPs. In aggregate, the κ values had a mean of 0.64, a standard deviation of 0.14, a median of 0.65, and a range of 0.27 to 0.91. Nine CTs exhibited good agreement in the majority of their pair-specific κ values with CPs (high-concordance CTs; 88 pair-specific κ values). This allowed us to identify outlier CPs who did not demonstrate good agreement with high-concordance CTs (16 of 88 pair-specific κ values [18.2%]). CONCLUSIONS: Laboratories can use this κ to determine when CP levels of agreement with CTs depart from those of their peers. Adding this to established metrics can give a more nuanced impression of CP performance.

AB - BACKGROUND: Although objective measures of cytotechnologist (CT) and cytopathologist (CP) performance exist, challenges remain. Two assumptions deserve examination: CPs' interpretations are correct, and CTs and CPs render interpretations independently of each other. This study presents a CT-CP interpretation comparison and provides insight into these assumptions. METHODS: Every gynecologic cytology specimen examined by both a CT and a CP from December 2004 to March 2015 was extracted from the laboratory information system; glandular interpretations were excluded. Excel and SAS were used for CT-CP pair analysis. CT-CP pairs with fewer than 32 specimens (the lowest quartile) were excluded. For the remaining CT-CP pairs, 30 specimens or 10% of the specimens (whichever was higher) were randomly selected for comparison by a weighted κ statistic. κ values greater than 0.6 represented good agreement within CT-CP pairs. RESULTS: This study evaluated 7116 of 53,241 gynecologic cytology specimens (13.4%) that received CT and CP interpretations. This resulted in 155 pair-specific κ values from 15 CTs and 16 CPs. In aggregate, the κ values had a mean of 0.64, a standard deviation of 0.14, a median of 0.65, and a range of 0.27 to 0.91. Nine CTs exhibited good agreement in the majority of their pair-specific κ values with CPs (high-concordance CTs; 88 pair-specific κ values). This allowed us to identify outlier CPs who did not demonstrate good agreement with high-concordance CTs (16 of 88 pair-specific κ values [18.2%]). CONCLUSIONS: Laboratories can use this κ to determine when CP levels of agreement with CTs depart from those of their peers. Adding this to established metrics can give a more nuanced impression of CP performance.

KW - Agreement

KW - Cytopathologist

KW - Cytotechnologist

KW - Gynecologic

KW - Outlier

KW - Papanicolaou test

KW - Performance

KW - Quality

KW - SQL

KW - κ

UR - http://www.scopus.com/inward/record.url?scp=85017207534&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85017207534&partnerID=8YFLogxK

U2 - 10.1002/cncy.21856

DO - 10.1002/cncy.21856

M3 - Article

JO - Cancer cytopathology

JF - Cancer cytopathology

SN - 1934-662X

ER -