Language adaptive state space learning with text guided pseudo supervision enables weakly supervised skin lesion segmentation

Accurate lesion segmentation in dermoscopy images is critical for high-throughput clinical screening, yet deploying automated diagnostic systems is severely hindered by the prohibitive computational cost of massive foundation models and the manual bottleneck of acquiring dense annotations. While recent vision-language pipelines alleviate the labeling burden by generating pseudo-masks from textual prompts, their staggering memory footprints and extreme inference latency render them unviable for real-time clinical edge deployment. Furthermore, contemporary State Space Models (SSMs) employed in such tasks typically process spatial features in fixed raster-scan orders, failing to efficiently capture semantic coherence between spatially disparate lesion regions. To overcome these computational and algorithmic bottlenecks, we propose a novel pseudo-supervision framework driven by a computational offloading paradigm. By isolating the massive multi-modal workloads to a highly parallelized offline pipeline, we strictly preserve real-time efficiency during online inference. The cornerstone of our method is an Adaptive Selective State-Space Module (ASSM) that employs a differentiable semantic routing mechanism to dynamically group related pixels into coherent sequences, ensuring robust long-range dependency modeling with minimal computational overhead. To harness textual priors without inflating inference latency, we introduce a Language-Fusion Squeeze-and-Excitation (LFSE) module to integrate domain-specific lesion descriptions via multi-scale cross-modal attention during the distributed training phase. The resulting semantic attention maps drive a Text-Guided Patch Augmentation (TPA) strategy to explicitly target uncertain boundary regions. By aligning a pure-vision student with a multimodal student and an exponential moving average teacher, our framework achieves robust segmentation relying solely on raw images during inference. Crucially, by entirely discarding the text-branch overhead, the deployed pure-vision model operates with exceptional computational efficiency. Specifically, it requires only 8.40M parameters and 0.37GB of peak memory, alongside a competitive computation cost of 18.06G FLOPs, perfectly satisfying the stringent real-time processing demands of modern clinical edge environments. Extensive experiments on five dermoscopic and endoscopic segmentation benchmarks demonstrate the strong cross-domain generalizability of the proposed approach. By utilizing bounding-box priors exclusively from a single source domain (ISIC 2017) to warm up the text-grounded detector, our pipeline operates in a strictly target-annotation-free regime for all other evaluated datasets. Compared to the most competitive text-supervised baseline, our method yields consistent absolute Dice Similarity Coefficient improvements of 1.41%, 1.65%, 1.38%, 1.64%, and 0.45% across the respective datasets, substantially narrowing the performance gap with fully supervised models while guaranteeing high-speed clinical applicability.