Uncertainty Aware Tri-Modal Fusion with Faithful Explainability for Five-Stage Alzheimer's Disease Classification

Early and accurate staging of Alzheimer's disease (AD) is a significant facet of prognosis, clinical trial participation and treatment selection, but remains difficult because single modality approaches apply each input equally and do not capture complimentary biomarkers. Presented with this problem, we proposed an uncertainty-aware tri-modal framework that combines, (i) IBSI compliant radiomic features obtained from the hippocampus, ventricles and cortex, (ii) 3D Swin-UNet transformer embeddings that encodes the multi-scale atrophy patterns, and (iii) fine-grained cognitive scores from the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA). Our model makes use of Monte-Carlo dropout to jointly estimate patient-specific epistemic uncertainty, which dynamically weights each mode of input through a gated self-attention mechanism, automatically eliminating the weight of noise from motion-corrupted Magnetic Resonance Imaging (MRI) or unreliable assessments measuring cognitive performance. The framework is evaluated on 1,080 individuals from ADNI, OASIS-3, and AIBL datasets and yields an average accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$88.6 \pm 1.2 \%$</tex>, and average Macro F1 and MacroAUC of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$86.4 \pm 1.3 \%$</tex> and 0.918 respectively across five AD stages, while also achieving 87.2 % accuracy on a completely independent multi-site test set. Importantly, these performance metrics are on par with or exceed state-of-the-art observation models, and are achieved using a low-cost T4 GPU in just <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.11 ~\mathrm{s} / \text{scan}$</tex>. Our model has a three-part explainability system using the transformers' saliency maps, the radiomic SHAP values, and the cognitive item attention weights, and demonstrates strong causal effort: perturbation of the voxels in the top- 10 % of saliency reduced confidence by 25 %, while masking the random regions had less than (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$&lt;5 \%$</tex>) effect on confidence. These results collectively indicate that uncertainty-aware multimodal fusion and rigorous validation for interpretability are imperative in creating clinically permissible AD stage systems.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.