Label Cleaning in Multimodal Public Retinal Image Datasets

Abstract Background Public datasets are widely used to develop and validate artificial intelligence models, but label noise may compromise model performance and benchmark reliability. This study systematically assessed and corrected label noise in public multimodal retinal image datasets using automated screening and structured expert verification. Methods Nine public retinal image datasets spanning color fundus photography, optical coherence tomography, and ultra-widefield fundus imaging were included. After preprocessing, 34 924 images were analyzed using Cleanlab’s Datalab module to identify near-duplicates, suspected label issues, and outlier samples. All flagged images underwent a four-stage manual verification process. Results Perceptual hash verification confirmed 638 near-duplicate images across six datasets. Cleanlab flagged 4 434 images (12.70%) for expert review, of which 1 884 (5.39% of all images) were confirmed as label errors. Dataset-level positive predictive values for label error detection ranged from 1.50% to 57.14%. Agreement between Cleanlab-suggested and expert-verified labels ranged from 51.95% to 100.00%, exceeding 60.00% in seven datasets. Among confirmed errors, 1 435 were in-distribution corrections, 103 required multi-label assignment, and 400 were out-of-distribution (OOD) cases, including rare diseases and poor-quality images. After accounting for overlap, 449 cases (23.83%) could not be represented within the original single-label classification framework. Conclusions Label noise is prevalent in public retinal image datasets and includes a substantial proportion of OOD and multi-label cases that cannot be fully addressed within conventional single-label classification frameworks. Cleanlab effectively reduced manual review burden and provided useful correction suggestions within the source taxonomy. This workflow enabled the creation of cleaner multimodal retinal datasets.