Does predictive accuracy reflect anatomical attention in CVM staging? evidence from explainable held-out vendor evaluation

BackgroundDeep learning has shown promising performance for cervical vertebral maturation (CVM) staging, but reported accuracies vary widely across studies. Differences in image preprocessing and imaging systems may affect model performance, while the black-box nature of deep learning makes it unclear whether predictions are based on clinically relevant cervical vertebral morphology.MethodsA total of 908 lateral cephalometric radiographs from four imaging vendors were retrospectively collected. Of these, 708 images from three vendors were used for model development and internal validation, while 200 images from a held-out vendor were reserved for external testing. ResNet-50 and Swin Transformer were evaluated across six preprocessing pipelines. Classification accuracy, Macro-F1, adjacent-stage accuracy, and severe misclassification were assessed. Grad-CAM and attention rollout were used for qualitative interpretability analysis.ResultsExternal exact-stage accuracy was consistently lower than internal performance. The highest external accuracies were 0.47 for ResNet-50 with min–max plus Z-score normalization and 0.42 for Swin Transformer with min–max plus CLAHE. Adjacent-stage accuracy was substantially higher, but severe misclassifications persisted. Preprocessing effects were architecture-dependent, with no single strategy consistently improving both models. Explainability maps showed variable attention alignment with C2–C4 and occasional emphasis on non-classical anatomical or imaging-dependent regions.ConclusionsDeep learning performance for CVM staging did not generalize consistently across imaging conditions, and preprocessing influenced different architectures differently. These findings suggest that predictions from machine learning–based CVM models should be interpreted with caution, particularly when applied to data acquired under different imaging conditions.