TY - JOUR
T1 - Machine learning does not improve upon traditional regression in predicting outcomes in atrial fibrillation
T2 - An analysis of the ORBIT-AF and GARFIELD-AF registries
AU - Loring, Zak
AU - Mehrotra, Suchit
AU - Piccini, Jonathan P.
AU - Camm, John
AU - Carlson, David
AU - Fonarow, Gregg C.
AU - Fox, Keith A.A.
AU - Peterson, Eric D.
AU - Pieper, Karen
AU - Kakkar, Ajay K.
N1 - Publisher Copyright:
© The Author(s) 2020.
PY - 2020/11/1
Y1 - 2020/11/1
N2 - Aims: Prediction models for outcomes in atrial fibrillation (AF) are used to guide treatment. While regression models have been the analytic standard for prediction modelling, machine learning (ML) has been promoted as a potentially superior methodology. We compared the performance of ML and regression models in predicting outcomes in AF patients. Methods and results: The Outcomes Registry for Better Informed Treatment of Atrial Fibrillation (ORBIT-AF) and Global Anticoagulant Registry in the FIELD (GARFIELD-AF) are population-based registries that include 74 792 AF patients. Models were generated from potential predictors using stepwise logistic regression (STEP), random forests (RF), gradient boosting (GB), and two neural networks (NNs). Discriminatory power was highest for death [STEP area under the curve (AUC) = 0.80 in ORBIT-AF, 0.75 in GARFIELD-AF] and lowest for stroke in all models (STEP AUC = 0.67 in ORBIT-AF, 0.66 in GARFIELD-AF). The discriminatory power of the ML models was similar or lower than the STEP models for most outcomes. The GB model had a higher AUC than STEP for death in GARFIELD-AF (0.76 vs. 0.75), but only nominally, and both performed similarly in ORBIT-AF. The multilayer NN had the lowest discriminatory power for all outcomes. The calibration of the STEP modelswere more aligned with the observed events for all outcomes. In the cross-registry models, the discriminatory power of the ML models was similar or lower than the STEP for most cases. Conclusion: When developed from two large, community-based AF registries, ML techniques did not improve prediction modelling of death, major bleeding, or stroke.
AB - Aims: Prediction models for outcomes in atrial fibrillation (AF) are used to guide treatment. While regression models have been the analytic standard for prediction modelling, machine learning (ML) has been promoted as a potentially superior methodology. We compared the performance of ML and regression models in predicting outcomes in AF patients. Methods and results: The Outcomes Registry for Better Informed Treatment of Atrial Fibrillation (ORBIT-AF) and Global Anticoagulant Registry in the FIELD (GARFIELD-AF) are population-based registries that include 74 792 AF patients. Models were generated from potential predictors using stepwise logistic regression (STEP), random forests (RF), gradient boosting (GB), and two neural networks (NNs). Discriminatory power was highest for death [STEP area under the curve (AUC) = 0.80 in ORBIT-AF, 0.75 in GARFIELD-AF] and lowest for stroke in all models (STEP AUC = 0.67 in ORBIT-AF, 0.66 in GARFIELD-AF). The discriminatory power of the ML models was similar or lower than the STEP models for most outcomes. The GB model had a higher AUC than STEP for death in GARFIELD-AF (0.76 vs. 0.75), but only nominally, and both performed similarly in ORBIT-AF. The multilayer NN had the lowest discriminatory power for all outcomes. The calibration of the STEP modelswere more aligned with the observed events for all outcomes. In the cross-registry models, the discriminatory power of the ML models was similar or lower than the STEP for most cases. Conclusion: When developed from two large, community-based AF registries, ML techniques did not improve prediction modelling of death, major bleeding, or stroke.
KW - Atrial fibrillation
KW - Machine learning
KW - Outcomes
UR - http://www.scopus.com/inward/record.url?scp=85096152737&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096152737&partnerID=8YFLogxK
U2 - 10.1093/europace/euaa172
DO - 10.1093/europace/euaa172
M3 - Article
C2 - 32879969
AN - SCOPUS:85096152737
SN - 1099-5129
VL - 22
SP - 1635
EP - 1644
JO - Europace
JF - Europace
IS - 11
ER -