OBJECTIVE: This study aims to conduct a comprehensive analysis of machine learning (ML) methods for MRI-based biomarker selection and classification to investigate early cognitive decline in AD. The focus to discriminate between classifying healthy control (HC) participants who remained stable and those who developed mild cognitive impairment (MCI) within five years (unstable HC or uHC).

METHODS: 3-Tesla (3T) MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) and Open Access Series of Imaging Studies 3 (OASIS-3) were used, focusing on HC and uHC groups. Freesurfer's recon-all and other tools were used to extract anatomical biomarkers from subcortical and cortical brain regions. ML techniques were applied for feature selection and classification, using the MATLAB Classification Learner (MCL) app for initial analysis, followed by advanced methods such as nested cross-validation and Bayesian optimization, which were evaluated within a Monte Carlo replication analysis as implemented in our customized pipeline. Additionally, polynomial regression-based data harmonization techniques were used to enhance ML and statistical analysis. In our study, ML classifiers were evaluated using performance metrics such as Accuracy (Acc), area under the receiver operating characteristic curve (AROC), F1-score, and a normalized Matthew's correlation coefficient (MCC').

RESULTS: Feature selection consistently identified biomarkers across ADNI and OASIS-3, with the entorhinal, hippocampus, lateral ventricle, and lateral orbitofrontal regions being the most affected. Classification results varied between balanced and imbalanced datasets and between ADNI and OASIS-3. For ADNI balanced datasets, the naíve Bayes model using z-score harmonization and ReliefF feature selection performed best (Acc = 69.17%, AROC = 77.73%, F1 = 69.21%, MCC' = 69.28%). For OASIS-3 balanced datasets, SVM with zscore-corrected data outperformed others (Acc = 66.58%, AROC = 72.01%, MCC' = 66.78%), while logistic regression had the best F1-score (66.68%). In imbalanced data, RUSBoost showed the strongest overall performance on ADNI (F1 = 50.60%, AROC = 81.54%) and OASIS-3 (MCC' = 63.31%). Support vector machine (SVM) excelled on ADNI in terms of Acc (82.93%) and MCC' (70.21%), while naïve Bayes performed best on OASIS-3 by F1 (42.54%) and AROC (70.33%).

APPROACH: Data from 324 participants (166 HC and 158 MCI) were analyzed using automated artifact rejection and source reconstruction. Spectral biomarkers, including sensor-space and sourcespace representations, were extracted from the magnetometer and gradiometer signals. Five inverse-solution methods were assessed (including linearly constrained minimum variance [LCMV] and exact low-resolution electromagnetic tomography analysis [eLORETA]) to increase the robustness of source-based biomarker selection. Multiple classifiers, including a support vector machine (SVM) and L1-regularized logistic regression (GLMNET), were compared using nested K-fold cross-validation embedded in Monte Carlo replications to limit overfitting.

MAIN RESULTS: The best performance was obtained by combining MRI features with source-based MEG features using GLMNET, achieving 76.31 ± 1.47% accuracy and 82.44 ± 1.44% AUC. MEG-only models showed strong discrimination and outperformed MRI-only analyses, particularly those based on the LCMV and eLORETA features. Across settings, the SVM and GLMNET were the most consistently effective classifiers.