Detecting Mild Cognitive Impairment (MCI) is a central problem of computational neurology owing to the heterogeneous nature of MCI presentations and the highly individualized nature of conversion trajectories. A mild cognitive impairment is a pre-stage of Alzheimer’s disease (AD). The current study proposes a Multimodal Transformer framework (MM-TF) that jointly assesses 3D structural MRI volumes, longitudinal cognitive scores (MMSE, ADAS-Cog, CDR), and genetic-demographic covariates (APOE ε4 status, age and sex) within a single cross-modal attention backbone. Each modality is tokenized by a modality-specific encoder a shared fusion transformer with multi-head co-attention learns cross-modal interactions which are routed to a four-way diagnostic head that is CN/EMCI/LMCI/AD and a conversion-risk head at 36-months. Assessed on 2,214 subjects from the public Alzheimer’s Disease Neuroimaging Initiative (ADNI), the framework achieves an illustrative four-way accuracy of 92.3% (macro-F1 0.921) and sMCI-vs-pMCI AUC of 0.902, outperforming CNN-based and early-fusion baselines in all tested configurations. A broad ablation reveals the contribution of cross-modal attention (+4.2 points relative to early concatenation) and of genetic covariates (+2.5). Numeric findings presented must be replaced with the authors’ actual results from experimental runs prior to submission.

We presented MM-TF, a multimodal transformer framework that converts structural MRI, longitudinal cognitive scores, and genetic-demographic covariates into a shared attention space to detect MCI and AD, as well as predict conversion after 36 months. Apart from accuracy in competitive staging, it is robust to missing modalities from a single trained model and calibrated uncertainty with out-of-distribution guarding. Further, there is also alignment of learned saliency with known AD neuropathology. The framework can be expanded with new modalities including CSF and DTI, as well as amyloid PET. Future biomarkers can be added as additional token streams without architectural change. Before submission, the authors must conduct real experiments on the ADNI download and replace every marked illustrative number.

[1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008. [2] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al., “An image is worth 16×16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learning Representations (ICLR), 2021. [3] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Zhang, Z. Li, et al., “Swin Transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2021, pp. 9992–10002. [4] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. [5] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. [6] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” in Proc. Int. Conf. Learning Representations (ICLR), 2015. [7] C. R. Jack, Jr., M. A. Bernstein, N. C. Fox, P. Thompson, G. Alexander, D. Harvey, et al., “The Alzheimer’s Disease Neuroimaging Initiative (ADNI): MRI methods,” J. Magn. Reson. Imag., vol. 27, no. 4, pp. 685–691, 2008. [8] M. F. Folstein, S. E. Folstein, and P. R. McHugh, “‘Mini-mental state’: A practical method for grading the cognitive state of patients for the clinician,” J. Psychiatric Res., vol. 12, no. 3, pp. 189–198, 1975. [9] G. McKhann, D. Drachman, M. Folstein, R. Katzman, D. Price, and E. M. Stadlan, “Clinical diagnosis of Alzheimer’s disease: Report of the NINCDS–ADRDA work group,” Neurology, vol. 34, no. 7, pp. 939–944, 1984. [10] R. Cuingnet, E. Gerardin, J. Tessieras, G. Auzias, S. Lehéricy, M.-O. Habert, et al., “Automatic classification of patients with Alzheimer’s disease from structural MRI: A review of the literature,” NeuroImage, vol. 56, no. 2, pp. 342–358, 2011. [11] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in Proc. Int. Conf. Learning Representations (ICLR), 2019. [12] T. Luong, H. Pham, and C. D. Manning, “Effective approaches to attention-based neural machine translation,” in Proc. EMNLP, 2015, pp. 1412–1421. [13] [VERIFY] A. Adeel, T. Zia, A. Zafar, et al., “Vision transformer based Alzheimer’s disease diagnostic system using MRI scans,” 2022. [14] [VERIFY] R. Venugopalan et al., “Transformer-based self-supervised learning for Alzheimer’s disease classification,” Frontiers in Aging Neuroscience, vol. 14, 2022. [15] [VERIFY] J. Sepahvand, M. J. Rezaei, and K. Maghooli, “Alzheimer’s disease diagnosis using adaptive 3D convolutional neural networks and MRI images,” Comput. Biol. Med., vol. 164, art. 107342, 2023. [16] [VERIFY] M. Huang and P.-T. Yap, “WS-Net: A cascaded weakly-supervised network for imbalanced Alzheimer’s disease staging,” IEEE Trans. Med. Imag., vol. 42, no. 5, pp. 1240–1252, 2023. [17] [VERIFY] L. Zhang et al., “Multimodal fusion transformer for Alzheimer’s disease progression prediction,” IEEE J. Biomed. Health Informat., 2023. [18] [VERIFY] P. J. LaMontagne, T. Benzinger, J. Morris, S. Keefe, R. Hornbeck, et al., “OASIS-3: Longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and Alzheimer’s disease,” medRxiv 2019.12.13.19014902, 2019. [19] [VERIFY] F. Dubois et al., “Deep learning for early Alzheimer’s disease detection using MRI and neuropsychological data: A systematic review,” Comput. Biol. Med., vol. 165, art. 107409, 2023. [20] [VERIFY] Y. Qiu et al., “Hierarchical multimodal fusion transformer for dementia staging,” Expert Syst. Appl., vol. 224, art. 120033, 2023. [21] [VERIFY] S. A. Al-garaibeh et al., “Early detection of Alzheimer’s disease using deep learning on MRI: A comparative study,” Diagnostics, vol. 13, no. 10, 2023. [22] [VERIFY] M. W. Weiner et al., “The Alzheimer’s Disease Neuroimaging Initiative: A review of ADNI publications,” Alzheimer’s & Dementia, vol. 13, no. 3, pp. 346–371, 2017. [23] [VERIFY] H. Lu et al., “Multimodal and multi-task deep learning for Alzheimer’s disease diagnosis and progression prediction,” Frontiers in Neuroscience, vol. 16, 2022. [24] [VERIFY] A. Padilla et al., “Self-supervised learning for MRI-based Alzheimer’s disease classification: A comparative study,” NeuroImage, vol. 274, 2023. [25] [VERIFY] K. N. Khan et al., “Cross-attention based multimodal fusion for neurological disorder classification,” IEEE Access, vol. 11, 2023. [26] Alzheimer’s Disease Neuroimaging Initiative. ADNI database. [Online]. Available: http://adni.loni.usc.edu (accessed Oct. 2026).