Exploring the behavior of hospital composite performance measures: An example from coronary artery bypass surgery

Sean M. O'Brien, Elizabeth R. Delong, Rachel S. Dokholyan, Fred H. Edwards, Eric D. Peterson

Research output: Contribution to journalArticlepeer-review

30 Scopus citations

Abstract

BACKGROUND - Composite scores that combine several performance measures into a single ranking are becoming the accepted metric for assessing hospital performance. In particular, the Centers for Medicare & Medicaid Services Hospital Quality Incentive Demonstration (HQID) project bases financial rewards and penalties on these scores. Although the HQID composite calculation is straightforward and easily understood, its method of combining process and outcome measures has not been validated. METHODS AND RESULTS - Using data on 530 hospitals from the Society of Thoracic Surgeons National Cardiac Database, we replicated the HQID methodology with 6 nationally endorsed performance measures (5 process measures plus survival) for coronary artery bypass surgery. Composite scores were essentially determined by process measure performance alone; the survival component explained only 4% of the composite score's total variance. This result persisted even when the survival component was allowed a 5-fold greater weighting in the composite summary. The popular "all-or-none" measurement approach was also dominated by the process component. Substantial disagreement was found among hospital rankings when several alternative methods were used; up to 60% of hospitals eligible for the top financial reward under HQID would change designation depending on the composite methodology used. The application of a simple statistical adjustment (standardization) to each method would provide more consistent results and a more balanced assessment of performance based on both process and outcomes. CONCLUSIONS - Existing methods used to create composite performance measures have remarkably different weighting of process versus outcomes metrics and lead to highly divergent provider rankings. Simple alternative methods can create more balanced process-outcome performance assessments.

Original languageEnglish (US)
Pages (from-to)2969-2975
Number of pages7
JournalCirculation
Volume116
Issue number25
DOIs
StatePublished - Dec 2007
Externally publishedYes

Keywords

  • Coronary artery bypass
  • Program evaluation
  • Quality of health care

ASJC Scopus subject areas

  • Cardiology and Cardiovascular Medicine
  • Physiology (medical)

Fingerprint

Dive into the research topics of 'Exploring the behavior of hospital composite performance measures: An example from coronary artery bypass surgery'. Together they form a unique fingerprint.

Cite this