Attention-Guided Multi-modal Deep Fusion Network for Early Prediction of Alzheimer’s Disease Using Structural MRI and PET Imaging

Alzheimer’s Disease (AD) is a neurodegenerative disorder characterized by progressive neurodegeneration, and it is one of the most common types of dementia worldwide. Being able to accurately and timely diagnosis AD remains a challenge in neuroscience. Our work presented in this study introduces a fully functional deep learning framework called Attention-Guided Multi-Modal Deep Fusion Network (AMDF-Net) that can help to predict the early diagnosis of Alzheimer’s disease using structural MRI (sMRI) and FDG-PET imaging data through an integrated model. In AMDF-Net, we utilize dual-branch 3D convolutional neural networks (CNNs) as well as Transformer encoders to extract local and global spatial dependencies within volumetric brain images or their associated volumetric datasets. The output of the subsequently fused data is more indicative of the spatial and unique relationship because of the integrated cross-modal attention mechanism, which concurrently aligns and optimises the salient disease-related information across both modalities. The AMDF-Net was compared against the current leading state-of-the-art multimodal models on the Alzheimer’s Disease. Neuroimaging Initiative (ADNI) dataset, attaining 92.1% accuracy, 95.4% AUC, and 0.91 F1 score. Explainability was established through Grad-CAM and Transformer attention maps, enabling clinicians to examine visually representative imaging of clinically relevant brain regions associated with the disease. These successful results indicate that attention-guided multi-modal deep learning should be investigated further as a way to address future clinical decisions around the accuracy and interpretability of diagnosis of Alzheimer’s disease derived from neuroimaging data.