Statistical uncertainty of mortality rates and rankings for children's hospitals

Chris Feudtner, Jay G. Berry, Gareth Parry, Paul Hain, Rustin B. Morse, Anthony D. Slonim, Samir S. Shah, Matt Hall

Research output: Contribution to journalArticle

14 Citations (Scopus)

Abstract

OBJECTIVES: Hospitals are being required to report publically their adjusted mortality rates, which are then being used to rank hospitals. Our objectives were to assess the statistical reliability of the determination of a hospital's adjusted mortality rate, of comparisons of that rate with the rates of other hospitals, and of the use of those rates to rank the hospitals. METHODS: A cross-sectional study of 473 383 patients discharged from 42 US children's hospitals in 2008 was performed. Hospital-specific observed/expected (O/E) mortality rate ratios and corresponding hospital rankings, with 95% confidence intervals (CIs), were examined. RESULTS: Hospitals' O/E mortality rate ratios exhibited wide 95% CIs, and no hospital was clearly distinguishable from the other hospitals' aggregated mean mortality performance. Only 2 hospitals' mortality performance fell outside the comparator hospitals' 95% CI. Those hospitals' 95% CIs overlapped with the overall comparator set's 95% CI, which suggests that there were no statistically significant hospital outliers. Fourteen (33.3%) of the 42 hospitals had O/E ratios that were not statistically different from being in the 95% CI of the top 10% of hospitals. Hospital-specific mortality rate rankings displayed even broader 95% CIs; the typical hospital had a 95% CI range that spanned 22 rank-order positions. CONCLUSION: Children's hospital-specific measures of adjusted mortality rate ratios and rankings have substantial amounts of statistical imprecision, which limits the usefulness of such measures for comparisons of quality of care.

Original languageEnglish (US)
JournalPediatrics
Volume128
Issue number4
DOIs
StatePublished - Oct 2011

Fingerprint

Uncertainty
Mortality
Confidence Intervals
Hospital Mortality
Ranking
Mortality Rate
Quality of Health Care
Confidence Interval

Keywords

  • Hospital performance
  • Mortality rates
  • Quality appraisal
  • Quality improvement

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Arts and Humanities (miscellaneous)

Cite this

Statistical uncertainty of mortality rates and rankings for children's hospitals. / Feudtner, Chris; Berry, Jay G.; Parry, Gareth; Hain, Paul; Morse, Rustin B.; Slonim, Anthony D.; Shah, Samir S.; Hall, Matt.

In: Pediatrics, Vol. 128, No. 4, 10.2011.

Research output: Contribution to journalArticle

Feudtner, Chris ; Berry, Jay G. ; Parry, Gareth ; Hain, Paul ; Morse, Rustin B. ; Slonim, Anthony D. ; Shah, Samir S. ; Hall, Matt. / Statistical uncertainty of mortality rates and rankings for children's hospitals. In: Pediatrics. 2011 ; Vol. 128, No. 4.
@article{867b1c7f61b546c98c6adb42dc8ce753,
title = "Statistical uncertainty of mortality rates and rankings for children's hospitals",
abstract = "OBJECTIVES: Hospitals are being required to report publically their adjusted mortality rates, which are then being used to rank hospitals. Our objectives were to assess the statistical reliability of the determination of a hospital's adjusted mortality rate, of comparisons of that rate with the rates of other hospitals, and of the use of those rates to rank the hospitals. METHODS: A cross-sectional study of 473 383 patients discharged from 42 US children's hospitals in 2008 was performed. Hospital-specific observed/expected (O/E) mortality rate ratios and corresponding hospital rankings, with 95{\%} confidence intervals (CIs), were examined. RESULTS: Hospitals' O/E mortality rate ratios exhibited wide 95{\%} CIs, and no hospital was clearly distinguishable from the other hospitals' aggregated mean mortality performance. Only 2 hospitals' mortality performance fell outside the comparator hospitals' 95{\%} CI. Those hospitals' 95{\%} CIs overlapped with the overall comparator set's 95{\%} CI, which suggests that there were no statistically significant hospital outliers. Fourteen (33.3{\%}) of the 42 hospitals had O/E ratios that were not statistically different from being in the 95{\%} CI of the top 10{\%} of hospitals. Hospital-specific mortality rate rankings displayed even broader 95{\%} CIs; the typical hospital had a 95{\%} CI range that spanned 22 rank-order positions. CONCLUSION: Children's hospital-specific measures of adjusted mortality rate ratios and rankings have substantial amounts of statistical imprecision, which limits the usefulness of such measures for comparisons of quality of care.",
keywords = "Hospital performance, Mortality rates, Quality appraisal, Quality improvement",
author = "Chris Feudtner and Berry, {Jay G.} and Gareth Parry and Paul Hain and Morse, {Rustin B.} and Slonim, {Anthony D.} and Shah, {Samir S.} and Matt Hall",
year = "2011",
month = "10",
doi = "10.1542/peds.2010-3074",
language = "English (US)",
volume = "128",
journal = "Pediatrics",
issn = "0031-4005",
publisher = "American Academy of Pediatrics",
number = "4",

}

TY - JOUR

T1 - Statistical uncertainty of mortality rates and rankings for children's hospitals

AU - Feudtner, Chris

AU - Berry, Jay G.

AU - Parry, Gareth

AU - Hain, Paul

AU - Morse, Rustin B.

AU - Slonim, Anthony D.

AU - Shah, Samir S.

AU - Hall, Matt

PY - 2011/10

Y1 - 2011/10

N2 - OBJECTIVES: Hospitals are being required to report publically their adjusted mortality rates, which are then being used to rank hospitals. Our objectives were to assess the statistical reliability of the determination of a hospital's adjusted mortality rate, of comparisons of that rate with the rates of other hospitals, and of the use of those rates to rank the hospitals. METHODS: A cross-sectional study of 473 383 patients discharged from 42 US children's hospitals in 2008 was performed. Hospital-specific observed/expected (O/E) mortality rate ratios and corresponding hospital rankings, with 95% confidence intervals (CIs), were examined. RESULTS: Hospitals' O/E mortality rate ratios exhibited wide 95% CIs, and no hospital was clearly distinguishable from the other hospitals' aggregated mean mortality performance. Only 2 hospitals' mortality performance fell outside the comparator hospitals' 95% CI. Those hospitals' 95% CIs overlapped with the overall comparator set's 95% CI, which suggests that there were no statistically significant hospital outliers. Fourteen (33.3%) of the 42 hospitals had O/E ratios that were not statistically different from being in the 95% CI of the top 10% of hospitals. Hospital-specific mortality rate rankings displayed even broader 95% CIs; the typical hospital had a 95% CI range that spanned 22 rank-order positions. CONCLUSION: Children's hospital-specific measures of adjusted mortality rate ratios and rankings have substantial amounts of statistical imprecision, which limits the usefulness of such measures for comparisons of quality of care.

AB - OBJECTIVES: Hospitals are being required to report publically their adjusted mortality rates, which are then being used to rank hospitals. Our objectives were to assess the statistical reliability of the determination of a hospital's adjusted mortality rate, of comparisons of that rate with the rates of other hospitals, and of the use of those rates to rank the hospitals. METHODS: A cross-sectional study of 473 383 patients discharged from 42 US children's hospitals in 2008 was performed. Hospital-specific observed/expected (O/E) mortality rate ratios and corresponding hospital rankings, with 95% confidence intervals (CIs), were examined. RESULTS: Hospitals' O/E mortality rate ratios exhibited wide 95% CIs, and no hospital was clearly distinguishable from the other hospitals' aggregated mean mortality performance. Only 2 hospitals' mortality performance fell outside the comparator hospitals' 95% CI. Those hospitals' 95% CIs overlapped with the overall comparator set's 95% CI, which suggests that there were no statistically significant hospital outliers. Fourteen (33.3%) of the 42 hospitals had O/E ratios that were not statistically different from being in the 95% CI of the top 10% of hospitals. Hospital-specific mortality rate rankings displayed even broader 95% CIs; the typical hospital had a 95% CI range that spanned 22 rank-order positions. CONCLUSION: Children's hospital-specific measures of adjusted mortality rate ratios and rankings have substantial amounts of statistical imprecision, which limits the usefulness of such measures for comparisons of quality of care.

KW - Hospital performance

KW - Mortality rates

KW - Quality appraisal

KW - Quality improvement

UR - http://www.scopus.com/inward/record.url?scp=80053491875&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80053491875&partnerID=8YFLogxK

U2 - 10.1542/peds.2010-3074

DO - 10.1542/peds.2010-3074

M3 - Article

VL - 128

JO - Pediatrics

JF - Pediatrics

SN - 0031-4005

IS - 4

ER -