TriDermNet-HAGO: A multi-branch token–edge–region fusion network with hierarchical gradient-orchestrated optimization for superior skin lesion analysis

Skin lesion segmentation and classification are two key issues in early melanoma detection and computer-assisted dermatological diagnostics. However, there are many difficulties in dermoscopic images, such as complex boundaries, diverse colors, low contrast, and various kinds of artifacts like hairs and air bubbles. To address these difficulties, this paper proposes a novel multi-branch network architecture named TriDermNet, which can jointly perform segmentation and classification through feature specialization and task interaction. The network architecture includes a Region branch, an Edge branch, and a Token branch. The Region branch is used to learn contextual features from images at multiple scales, and the Edge branch is used to refine segmentation boundaries and remove artifacts. The Token branch is used to obtain global diagnostic evidence from images. The proposed network architecture is evaluated on three skin lesion datasets: ISIC 2016, ISIC 2017, and ISIC 2018. The experimental results show that TriDermNet achieves competitive baseline performance in skin lesion segmentation and classification, with a Dice score of 95.12%, 94.38%, and 95.47% on ISIC 2016, ISIC 2017, and ISIC 2018 datasets, respectively, and a classification AUC higher than 97.6%. The proposed TriDermNet framework provides a reliable solution to skin cancer screening. To improve the convergence instability, task imbalance, and local optima issues in the backpropagation process, this paper proposes a novel hierarchical adaptive gradient-orchestrated optimization algorithm named HAGO, in which a three-level hierarchical gradient regulation mechanism is used to stabilize the update process in each layer, balance gradients from multiple tasks, and introduce perturbations to avoid local optima. The proposed TriDermNet-HAGO framework achieves better segmentation and classification accuracy on ISIC 2016, ISIC 2017, and ISIC 2018 datasets with a Dice score of 96.63%, 95.81%, 96.24%, respectively and a classification AUC higher than 98.7%.