Multimodal Integration of Ambulatory ECG and Clinical Features for Sudden Cardiac Death and Pump Failure Death Prediction

Ambulatory electrocardiograms (ECGs) provide continuous monitoring of cardiac electrical activity, but many machine learning approaches analyze these recordings without incorporating broader patient clinical context. We evaluated a multimodal framework integrating ambulatory ECG-derived representations with clinical information for four-year prediction of sudden cardiac death (SCD) and pump failure death (PFD). The analysis included 730 patients from the MUSIC cohort, comprising 577 patients with no cardiac death, 71 with SCD, and 82 with PFD. Ambulatory ECG recordings were segmented and encoded using a multiple instance learning--temporal convolutional neural network, while clinical variables were represented as LLM-generated text, deterministic text, or tabular features. Models were developed and evaluated using patient-level nested cross-validation with five outer and four inner folds, with preprocessing, model and fusion-architecture selection, calibration, and threshold selection restricted to training data. For SCD, ECG + deterministic text achieved the highest AUROC of 0.711 (95% CI, 0.643-0.773), while ECG + tabular clinical features achieved the highest PR-AUC of 0.248 (0.180-0.357). For PFD, the unimodal tabular model achieved the highest AUROC of 0.787 (0.732-0.839) and PR-AUC of 0.360 (0.275-0.470), while ECG + tabular features achieved an AUROC of 0.756 (0.697-0.811) and PR-AUC of 0.353 (0.261-0.461). ECG + LLM text improved upon LLM text alone but did not outperform the unimodal ECG model. Attribution and token-masking analyses provided complementary evidence that model explanations were partly robust to superficial input perturbations while remaining sensitive to model and masking conditions, and representation and patient-text permutation analyses showed that predictive value depended strongly on how clinical information was represented. These findings indicate that the benefit of multimodal integration is outcome- and representation-dependent, with deterministic and tabular clinical representations providing greater predictive value than full LLM-generated assessments.