A dual backbone deep learning framework for early Alzheimer’s disease detection via multimodal handwriting analysis

Early detection of Alzheimer’s disease may benefit from low-cost, non-invasive screening tools. Handwriting is examined as a marker of cognitive and neuromotor decline by combining two image representations: offline ink-trace scans and online-derived RGB images. Offline images capture the final written product, while RGB images encode pressure, velocity, and jerk, including in-air movements near the writing surface. A dual-backbone ConvNeXt-S framework is proposed, in which each modality is processed by a separate encoder prior to normalisation, feature fusion, and refinement with a Convolutional Block Attention Module. Subject-wise five-fold cross-validation was conducted on 174 participants completing 25 handwriting tasks, with all samples from each participant kept within a single fold. Metrics were computed at the task-sample level. The offline-only model achieved an accuracy of 0.789 ± 0.043 and an MCC of 0.581 ± 0.090. The best multimodal model achieved the highest mean performance, with accuracy 0.799 ± 0.042 and MCC 0.601 ± 0.082, although the MCC gain over the offline-only baseline was small and not statistically significant (Wilcoxon p = 0.156 ; bootstrap p = 0.052 ). Grad-CAM maps indicated branch-specific regions, but these results remain qualitative. Offline handwriting provided the strongest signal, while RGB information served as a secondary cue within a modality-specific architecture.