Explainable Multimodal Ensemble Learning with a Secure Cloud Data Pipeline for District-Level Tuberculosis and Pneumonia Surveillance

Tuberculosis (TB) and pneumonia are difficult to separate on chest radiographs and remain under-diagnosed where radiologists are scarce. This paper presents an explainable multimodal framework that fuses chest X-ray (CXR) analysis with 18 clinical and demographic variables and connects it to a secure, role-based surveillance platform. Features from DenseNet121 (1,024-d) and EfficientNetB0 (1,280-d) are concatenated and classified by a multilayer perceptron; a second network processes the clinical data; and a completeness-aware late fusion gives the clinical branch up to 35% of the final weight in proportion to the share of clinical fields supplied, falling back to image-only inference when none are given. The clinical variables are synthetic records paired with public radiographs, so the multimodal results are a proof of concept rather than clinical evidence. On a held-out stratified test set (20% of the data), the ensemble reached an accuracy of 91.7%, a macro-averaged F1-score of 0.918 and an AUC-ROC of 0.958, compared with accuracies of 87.2% and 82.6% (AUC-ROC 0.921 and 0.872) for the image-only and clinical-only models. Grad-CAM heatmaps accompany every image-based prediction. Patient records are protected by field-level AES-256-GCM encryption, PBKDF2-HMAC-SHA256 key derivation, SHA-256 pseudonymised identifiers and HMAC-SHA256 integrity tags, and are persisted in Firebase Firestore. Streamlit dashboards serve patients, hospital administrators and government officials with district-level analytics, maps and threshold-based e-mail alerts. End-to-end inference took 1.4–8.8 s per case across three hardware tiers, and a usability study with 28 health professionals produced a System Usability Scale score of 83.4. We discuss architectural decisions, security trade-offs and the limitations that must be addressed before clinical deployment.