Leakage-Controlled and Calibrated Deep Ensemble Learning for Reliable COVID-19 Classification from Chest X-Ray and CT Images

Reliable evaluation of artificial intelligence models for coronavirus disease 2019 (COVID-19) imaging remains challenging because patient-level dependencies, source-related biases, and differences between imaging modalities can lead to overly optimistic performance estimates. This study proposes a leakage-controlled, modality-aware deep ensemble framework for COVID-19 classification using chest X-ray (CXR) and computed tomography (CT) images from the COVID-19 Radiography Database and the SARS-CoV-2 CT-Scan Dataset. Separate evaluation protocols were established for CXR and CT, with duplicate control and available source information incorporated for CXR and patient-level partitioning applied to CT before data augmentation. Four ImageNet-pretrained convolutional neural networks (CNNs)—VGG16, ResNet50, DenseNet169, and InceptionV3—were evaluated, followed by temperature scaling for probability calibration and validation-optimized weighted soft voting for ensemble fusion. The original CT architectural comparison was conducted at the slice level, whereas the final CT evaluation was performed at the patient level. In the final modality-specific held-out evaluation, DenseNet169 achieved an area under the receiver operating characteristic curve (AUROC) of 0.985, an accuracy of 97.32%, and an F1-score of 98.1% on the 2870-image CXR test set. The calibrated weighted ensemble achieved an AUROC of 0.990, an accuracy of 98.43%, and an F1-score of 97.9%, with a Brier score of 0.030 and an expected calibration error (ECE) of 0.023. For CT, DenseNet169 achieved an AUROC of 0.941 and correctly classified 16 of 18 patients (88.89%), whereas the calibrated weighted ensemble achieved an AUROC of 0.969 and correctly classified 17 of 18 patients (94.44%), with a Brier score of 0.070 and an ECE of 0.053. An uncertainty-aware selective-prediction analysis identified 316 of 2870 CXR test images (11.01%) and 2 of 18 CT test patients (11.11%) as low-confidence cases for specialist review. Among the remaining CT cases, 15 of 16 patients were correctly classified (93.75%). Because the CT test cohort contained only 18 patients, the patient-level CT performance and selective-prediction results should be interpreted cautiously as descriptive estimates from this limited cohort. The original pooled evaluation, in which DenseNet169 achieved 98.51% accuracy, is retained only as an exploratory architectural comparison and should not be interpreted as directly comparable with the final modality-specific evaluation. The findings demonstrate that evaluation design, statistical unit, probability calibration, and ensemble fusion materially influence the interpretation of COVID-19 imaging-classification performance. The proposed framework provides leakage-aware and calibrated performance estimates under the evaluated datasets; however, independent multicenter external validation is required before broader claims of clinical generalizability or deployment readiness can be made.