FUNDING: This research was supported by the United Kingdom Global Better Health Programme, which is managed by the United Kingdom Foreign, Commonwealth and Development Office and supported by PricewaterhouseCoopers in Southeast Asia.
OBJECTIVE: This study aims to conduct a comprehensive analysis of machine learning (ML) methods for MRI-based biomarker selection and classification to investigate early cognitive decline in AD. The focus to discriminate between classifying healthy control (HC) participants who remained stable and those who developed mild cognitive impairment (MCI) within five years (unstable HC or uHC).
METHODS: 3-Tesla (3T) MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and Open Access Series of Imaging Studies 3 (OASIS-3) were used, focusing on HC and uHC groups. Freesurfer's recon-all and other tools were used to extract anatomical biomarkers from subcortical and cortical brain regions. ML techniques were applied for feature selection and classification, using the MATLAB Classification Learner (MCL) app for initial analysis, followed by advanced methods such as nested cross-validation and Bayesian optimization, which were evaluated within a Monte Carlo replication analysis as implemented in our customized pipeline. Additionally, polynomial regression-based data harmonization techniques were used to enhance ML and statistical analysis. In our study, ML classifiers were evaluated using performance metrics such as Accuracy (Acc), area under the receiver operating characteristic curve (AROC), F1-score, and a normalized Matthew's correlation coefficient (MCC').
RESULTS: Feature selection consistently identified biomarkers across ADNI and OASIS-3, with the entorhinal, hippocampus, lateral ventricle, and lateral orbitofrontal regions being the most affected. Classification results varied between balanced and imbalanced datasets and between ADNI and OASIS-3. For ADNI balanced datasets, the naíve Bayes model using z-score harmonization and ReliefF feature selection performed best (Acc = 69.17%, AROC = 77.73%, F1 = 69.21%, MCC' = 69.28%). For OASIS-3 balanced datasets, SVM with zscore-corrected data outperformed others (Acc = 66.58%, AROC = 72.01%, MCC' = 66.78%), while logistic regression had the best F1-score (66.68%). In imbalanced data, RUSBoost showed the strongest overall performance on ADNI (F1 = 50.60%, AROC = 81.54%) and OASIS-3 (MCC' = 63.31%). Support vector machine (SVM) excelled on ADNI in terms of Acc (82.93%) and MCC' (70.21%), while naïve Bayes performed best on OASIS-3 by F1 (42.54%) and AROC (70.33%).
CONCLUSION: Data harmonization significantly improved the consistency and performance of feature selection and ML classification, with z-score harmonization yielding the best results. This study also highlights the importance of nested cross-validation (CV) to control overfitting and the potential of a semi-automatic pipeline for early AD detection using MRI, with future applications integrating other neuroimaging data to enhance prediction.