SkinNet-X: a task coupled feature interaction framework for joint skin lesion segmentation and classification

Well trained segmentation and classification of skin lesions from dermoscopic images are crucial to an early detection of melanoma and other malignant skin disease. While many deep learning models reach impressive overall performance, most existing approaches treat segmentation and classification as separate tasks, leading to limited interaction between spatial lesion localization and semantic disease knowledge. This segregation results in a high variance prediction and would reduce the overall robustness of automated diagnostic systems. To mitigate this limitation, we proposed SkinNet-X: a unified multitask framework to reformulate the skin lesion segmentation and classification tasks jointly. We achieve this with a shared hierarchical feature representation to transfer knowledge between both tasks while introducing new Cross Task Consistency Learning (CTCL) strategy that enforces semantic consistency between lesion localization and disease prediction during joint optimization. In order to enhance the discriminative capability of feature expression and alleviate redundant representations, a dual stage multi label feature optimization module is further exploited, accompanied by an adaptive PolyLoss based optimization strategy stabilizing multitask learning. The effectiveness of the proposed SkinNet-X framework was evaluated using segmentation and classification experiments on the ISIC 2018, ISIC 2019, and HAM10000 benchmark datasets. We also performed additional analyses, namely ablation studies, computational complexity evaluation, sensitivity analysis and statistical validation to explore the contribution of each architectural component. Experimental results show that the proposed multitask learning strategy can consistently improve accuracy of lesion localization and performance of classification with robust generalizability across multiple datasets. These findings demonstrate that SkinNet-X is an effective and potentially useful framework for automated skin lesion analysis and computer-aided dermatological diagnosis.