Statistical Validation Based on Parametric Receiver Operating Characteristic Analysis of Continuous Classification Data

Kelly H. Zou, Simon K. Warfield, Julia R. Fielding, Clare M.C. Tempany, William M. Wells, Michael R. Kaus, Ferenc A. Jolesz, Ron Kikinis

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Rationale and Objectives. The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples. Materials and Methods. Two parametric models were developed for diagnostic or imaging data. Example 1: A semiautomated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a nonlinear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by binormal distributions. In all examples, areas under the receiver operating characteristic curves were computed. Results. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnefic resonance imaging of brain tumors: meningiomas (0.924-0.984); astrocytomas (0.786-0.986); and other low-grade gliomas (0.896-0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific anfigen for staging prostate cancer (0.768). Conclusion. All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.

Original languageEnglish (US)
Pages (from-to)1359-1368
Number of pages10
JournalAcademic Radiology
Volume10
Issue number12
DOIs
StatePublished - Jan 1 2003

Fingerprint

ROC Curve
Diagnostic Imaging
Prostate-Specific Antigen
Brain Neoplasms
Spiral Computed Tomography
Astrocytoma
Meningioma
Prostatectomy
Routine Diagnostic Tests
Glioma
Prostate
Neoplasms
Prostatic Neoplasms
Magnetic Resonance Imaging
Clinical Trials
Therapeutics

Keywords

  • Brain segmentation
  • Computed tomography
  • Genitourinary system
  • Magnetic resonance
  • Prostate specific antigen (PSA)
  • Receiver operating characteristic (ROC) analysis

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

Statistical Validation Based on Parametric Receiver Operating Characteristic Analysis of Continuous Classification Data. / Zou, Kelly H.; Warfield, Simon K.; Fielding, Julia R.; Tempany, Clare M.C.; Wells, William M.; Kaus, Michael R.; Jolesz, Ferenc A.; Kikinis, Ron.

In: Academic Radiology, Vol. 10, No. 12, 01.01.2003, p. 1359-1368.

Research output: Contribution to journalArticle

Zou, Kelly H. ; Warfield, Simon K. ; Fielding, Julia R. ; Tempany, Clare M.C. ; Wells, William M. ; Kaus, Michael R. ; Jolesz, Ferenc A. ; Kikinis, Ron. / Statistical Validation Based on Parametric Receiver Operating Characteristic Analysis of Continuous Classification Data. In: Academic Radiology. 2003 ; Vol. 10, No. 12. pp. 1359-1368.
@article{d897dba8cb074de594c64c2ef3761cae,
title = "Statistical Validation Based on Parametric Receiver Operating Characteristic Analysis of Continuous Classification Data",
abstract = "Rationale and Objectives. The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples. Materials and Methods. Two parametric models were developed for diagnostic or imaging data. Example 1: A semiautomated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a nonlinear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by binormal distributions. In all examples, areas under the receiver operating characteristic curves were computed. Results. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnefic resonance imaging of brain tumors: meningiomas (0.924-0.984); astrocytomas (0.786-0.986); and other low-grade gliomas (0.896-0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific anfigen for staging prostate cancer (0.768). Conclusion. All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.",
keywords = "Brain segmentation, Computed tomography, Genitourinary system, Magnetic resonance, Prostate specific antigen (PSA), Receiver operating characteristic (ROC) analysis",
author = "Zou, {Kelly H.} and Warfield, {Simon K.} and Fielding, {Julia R.} and Tempany, {Clare M.C.} and Wells, {William M.} and Kaus, {Michael R.} and Jolesz, {Ferenc A.} and Ron Kikinis",
year = "2003",
month = "1",
day = "1",
doi = "10.1016/S1076-6332(03)00538-5",
language = "English (US)",
volume = "10",
pages = "1359--1368",
journal = "Academic Radiology",
issn = "1076-6332",
publisher = "Elsevier USA",
number = "12",

}

TY - JOUR

T1 - Statistical Validation Based on Parametric Receiver Operating Characteristic Analysis of Continuous Classification Data

AU - Zou, Kelly H.

AU - Warfield, Simon K.

AU - Fielding, Julia R.

AU - Tempany, Clare M.C.

AU - Wells, William M.

AU - Kaus, Michael R.

AU - Jolesz, Ferenc A.

AU - Kikinis, Ron

PY - 2003/1/1

Y1 - 2003/1/1

N2 - Rationale and Objectives. The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples. Materials and Methods. Two parametric models were developed for diagnostic or imaging data. Example 1: A semiautomated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a nonlinear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by binormal distributions. In all examples, areas under the receiver operating characteristic curves were computed. Results. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnefic resonance imaging of brain tumors: meningiomas (0.924-0.984); astrocytomas (0.786-0.986); and other low-grade gliomas (0.896-0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific anfigen for staging prostate cancer (0.768). Conclusion. All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.

AB - Rationale and Objectives. The accuracy of diagnostic test and imaging segmentation is important in clinical practice because it has a direct impact on therapeutic planning. Statistical validations of classification accuracy was conducted based on parametric receiver operating characteristic analysis, illustrated on three radiologic examples. Materials and Methods. Two parametric models were developed for diagnostic or imaging data. Example 1: A semiautomated fractional segmentation algorithm was applied to magnetic resonance imaging of nine cases of brain tumors. The tumor and background pixel data were assumed to have bi-beta distributions. Fractional segmentation was validated against an estimated composite pixel-wise gold standard based on multi-reader manual segmentations. Example 2: The predictive value of 100 cases of spiral computed tomography of ureteral stone sizes, distributed as bi-normal after a nonlinear transformation, under two treatment options received. Example 3: One hundred eighty cases had prostate-specific antigen levels measured in a prospective clinical trial. Radical prostatectomy was performed in all to provide a binary gold standard of local and advanced cancer stages. Prostate-specific antigen level was transformed and modeled by binormal distributions. In all examples, areas under the receiver operating characteristic curves were computed. Results. The areas under the receiver operating characteristic curves were: Example 1: Fractional segmentation of magnefic resonance imaging of brain tumors: meningiomas (0.924-0.984); astrocytomas (0.786-0.986); and other low-grade gliomas (0.896-0.983). Example 3: Ureteral stone size for treatment planning (0.813). Example 2: Prostate-specific anfigen for staging prostate cancer (0.768). Conclusion. All clinical examples yielded fair to excellent accuracy. The validation metric area under the receiver operating characteristic curves may be generalized to evaluating the performances of several continuous classifiers related to imaging.

KW - Brain segmentation

KW - Computed tomography

KW - Genitourinary system

KW - Magnetic resonance

KW - Prostate specific antigen (PSA)

KW - Receiver operating characteristic (ROC) analysis

UR - http://www.scopus.com/inward/record.url?scp=0345991249&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0345991249&partnerID=8YFLogxK

U2 - 10.1016/S1076-6332(03)00538-5

DO - 10.1016/S1076-6332(03)00538-5

M3 - Article

VL - 10

SP - 1359

EP - 1368

JO - Academic Radiology

JF - Academic Radiology

SN - 1076-6332

IS - 12

ER -