Reinforcement learning evaluation of treatment policies for patients with hepatitis C virus

Brandon Oselio, Amit G. Singal, Xuefei Zhang, Tony Van, Boang Liu, Ji Zhu, Akbar K. Waljee

Research output: Contribution to journalArticlepeer-review

Abstract

Background: Evaluation of new treatment policies is often costly and challenging in complex conditions, such as hepatitis C virus (HCV) treatment, or in limited-resource settings. We sought to identify hypothetical policies for HCV treatment that could best balance the prevention of cirrhosis while preserving resources (financial or otherwise). Methods: The cohort consisted of 3792 HCV-infected patients without a history of cirrhosis or hepatocellular carcinoma at baseline from the national Veterans Health Administration from 2015 to 2019. To estimate the efficacy of hypothetical treatment policies, we utilized historical data and reinforcement learning to allow for greater flexibility when constructing new HCV treatment strategies. We tested and compared four new treatment policies: a simple stepwise policy based on Aspartate Aminotransferase to Platelet Ratio Index (APRI), a logistic regression based on APRI, a logistic regression on multiple longitudinal and demographic indicators that were prespecified for clinical significance, and a treatment policy based on a risk model developed for HCV infection. Results: The risk-based hypothetical treatment policy achieved the lowest overall risk with a score of 0.016 (90% CI 0.016, 0.019) while treating the most high-risk (346.4 ± 1.4) and the fewest low-risk (361.0 ± 20.1) patients. Compared to hypothetical treatment policies that treated approximately the same number of patients (1843.7 vs. 1914.4 patients), the risk-based policy had more untreated time per patient (7968.4 vs. 7742.9 patient visits), signaling cost reduction for the healthcare system. Conclusions: Off-policy evaluation strategies are useful to evaluate hypothetical treatment policies without implementation. If a quality risk model is available, risk-based treatment strategies can reduce overall risk and prioritize patients while reducing healthcare system costs.

Original languageEnglish (US)
Article number63
JournalBMC Medical Informatics and Decision Making
Volume22
Issue number1
DOIs
StatePublished - Dec 2022

Keywords

  • Cirrhosis
  • Hepatology
  • Machine learning
  • Prediction modeling
  • Reinforcement learning
  • Risk-based treatment
  • Treatment policy

ASJC Scopus subject areas

  • Health Policy
  • Health Informatics

Fingerprint

Dive into the research topics of 'Reinforcement learning evaluation of treatment policies for patients with hepatitis C virus'. Together they form a unique fingerprint.

Cite this