OBJECTIVE: To derive a single algorithm using deep learning and machine learning for the prediction and identification of factors associated with in-hospital mortality in Asian patients with ACS and to compare performance to a conventional risk score.
METHODS: The Malaysian National Cardiovascular Disease Database (NCVD) registry, is a multi-ethnic, heterogeneous database spanning from 2006-2017. It was used for in-hospital mortality model development with 54 variables considered for patients with STEMI and Non-STEMI (NSTEMI). Mortality prediction was analyzed using feature selection methods with machine learning algorithms. Deep learning algorithm using features selected from machine learning was compared to Thrombolysis in Myocardial Infarction (TIMI) score.
RESULTS: A total of 68528 patients were included in the analysis. Deep learning models constructed using all features and selected features from machine learning resulted in higher performance than machine learning and TIMI risk score (p < 0.0001 for all). The best model in this study is the combination of features selected from the SVM algorithm with a deep learning classifier. The DL (SVM selected var) algorithm demonstrated the highest predictive performance with the least number of predictors (14 predictors) for in-hospital prediction of STEMI patients (AUC = 0.96, 95% CI: 0.95-0.96). In NSTEMI in-hospital prediction, DL (RF selected var) (AUC = 0.96, 95% CI: 0.95-0.96, reported slightly higher AUC compared to DL (SVM selected var) (AUC = 0.95, 95% CI: 0.94-0.95). There was no significant difference between DL (SVM selected var) algorithm and DL (RF selected var) algorithm (p = 0.5). When compared to the DL (SVM selected var) model, the TIMI score underestimates patients' risk of mortality. TIMI risk score correctly identified 13.08% of the high-risk patient's non-survival vs 24.7% for the DL model and 4.65% vs 19.7% of the high-risk patient's non-survival for NSTEMI. Age, heart rate, Killip class, cardiac catheterization, oral hypoglycemia use and antiarrhythmic agent were found to be common predictors of in-hospital mortality across all ML feature selection models in this study. The final algorithm was converted into an online tool with a database for continuous data archiving for prospective validation.
CONCLUSIONS: ACS patients were better classified using a combination of machine learning and deep learning in a multi-ethnic Asian population when compared to TIMI scoring. Machine learning enables the identification of distinct factors in individual Asian populations to improve mortality prediction. Continuous testing and validation will allow for better risk stratification in the future, potentially altering management and outcomes.
METHODS: Four validation groups were extracted from a longitudinal community-based study dataset of 12,573 participants aged ≥18 years to validate the Framingham Risk Score (FRS), Systematic COronary Risk Evaluation 2 (SCORE2), Revised Pooled Cohort Equations (RPCE), and World Health Organization cardiovascular disease (WHO CVD) models. Two measures of validation are examined: discrimination and calibration. Outcome of interest was 10-year risk of CVD events (fatal and non-fatal). SCORE2 and RPCE performances were compared to SCORE and PCE, respectively.
FINDINGS: FRS (AUC = 0.750) and RPCE (AUC = 0.752) showed good discrimination in CVD risk prediction. Although FRS and RPCE have poor calibration, FRS demonstrates smaller discordance for FRS vs. RPCE (298% vs. 733% in men, 146% vs. 391% in women). Other models had reasonable discrimination (AUC = 0.706-0.732). Only SCORE2-Low, -Moderate and -High (aged <50) had good calibration (X2 goodness-of-fit, P-value = 0.514, 0.189, 0.129, respectively). SCORE2 and RPCE showed improvements compared to SCORE (AUC = 0.755 vs. 0.747, P-value <0.001) and PCE (AUC = 0.752 vs. 0.546, P-value <0.001), respectively. Almost all risk models overestimated 10-year CVD risk by 3%-1430%.
INTERPRETATION: In Malaysians, RPCE are evaluated be the most clinically useful to predict CVD risk. Additionally, SCORE2 and RPCE outperformed SCORE and PCE, respectively.
FUNDING: This work was supported by the Malaysian Ministry of Science, Technology, and Innovation (MOSTI) (Grant No: TDF03211036).
OBJECTIVE: To employ machine learning (ML) and stacked ensemble learning (EL) methods in predicting short- and long-term mortality in Asian patients diagnosed with NSTEMI/UA and to identify the associated features, subsequently evaluating these findings against established risk scores.
METHODS: We analyzed data from the National Cardiovascular Disease Database for Malaysia (2006-2019), representing a diverse NSTEMI/UA Asian cohort. Algorithm development utilized in-hospital records of 9,518 patients, 30-day data from 7,133 patients, and 1-year data from 7,031 patients. This study utilized 39 features, including demographic, cardiovascular risk, medication, and clinical features. In the development of the stacked EL model, four base learner algorithms were employed: eXtreme Gradient Boosting (XGB), Support Vector Machine (SVM), Naive Bayes (NB), and Random Forest (RF), with the Generalized Linear Model (GLM) serving as the meta learner. Significant features were chosen and ranked using ML feature importance with backward elimination. The predictive performance of the algorithms was assessed using the area under the curve (AUC) as a metric. Validation of the algorithms was conducted against the TIMI for NSTEMI/UA using a separate validation dataset, and the net reclassification index (NRI) was subsequently determined.
RESULTS: Using both complete and reduced features, the algorithm performance achieved an AUC ranging from 0.73 to 0.89. The top-performing ML algorithm consistently surpassed the TIMI risk score for in-hospital, 30-day, and 1-year predictions (with AUC values of 0.88, 0.88, and 0.81, respectively, all p < 0.001), while the TIMI scores registered significantly lower at 0.55, 0.54, and 0.61. This suggests the TIMI score tends to underestimate patient mortality risk. The net reclassification index (NRI) of the best ML algorithm for NSTEMI/UA patients across these periods yielded an NRI between 40-60% (p < 0.001) relative to the TIMI NSTEMI/UA risk score. Key features identified for both short- and long-term mortality included age, Killip class, heart rate, and Low-Molecular-Weight Heparin (LMWH) administration.
CONCLUSIONS: In a broad multi-ethnic population, ML approaches outperformed conventional TIMI scoring in classifying patients with NSTEMI and UA. ML allows for the precise identification of unique characteristics within individual Asian populations, improving the accuracy of mortality predictions. Continuous development, testing, and validation of these ML algorithms holds the promise of enhanced risk stratification, thereby revolutionizing future management strategies and patient outcomes.