OphthUS-GPT: a multimodal AI system for ophthalmic B-scan ultrasound report generation and interpretation

Ophthalmic B-scan ultrasound plays a critical role in diagnosing posterior segment ocular diseases, yet traditional manual report generation is time-consuming and relies heavily on clinician expertise. In this study, we propose OphthUS-GPT, a multimodal artificial intelligence system designed to automatically generate diagnostic reports from ophthalmic B-scan ultrasound images and provide interactive support for report interpretation through human-machine dialogue. A retrospective analysis was conducted at the Affiliated Eye Hospital, Jiangxi Medical College, Nanchang University, encompassing 103,237 ultrasound images and 51,618 diagnostic reports from 51,618 patients. The BLIP (Bootstrapping Language-Image Pre-training) model was fine-tuned in a two-stage framework to generate diagnostic reports: stage (i) produced clinical findings, and stage (ii) integrated findings with clinical impressions. The DeepSeek large language model was incorporated to enhance report interpretability through multi-turn question-answering interactions. System performance was evaluated using text similarity metrics (BLEU, ROUGE, CIDEr), disease classification metrics (accuracy, sensitivity, specificity, F1 score), and expert ratings for clinical accuracy and completeness. The question-answering module was assessed on accuracy, completeness, security, and user satisfaction. Results demonstrate that OphthUS-GPT achieved a BLEU-1 score of 0.5739, ROUGE-L of 0.6131, and CIDEr of 0.9818 for report generation. For the majority of common conditions, including vitreous opacities, retinal detachment, posterior scleral staphyloma, and cataracts, classification accuracy exceeded 90%, while overall accuracy exceeded 80% across all evaluated disease categories. More than 90% of the generated reports received accuracy and completeness scores of 3 or higher from expert evaluators. The interactive question-answering module, particularly the locally deployed DeepSeek model, demonstrated statistically superior performance in accuracy compared to Claude, and in satisfaction compared to both Claude and GPT-4 Turbo, while no significant differences were observed across models in completeness and security. In conclusion, OphthUS-GPT represents a promising deep learning framework that integrates multimodal learning and large language models to automate ophthalmic ultrasound report generation while providing interpretable, interactive clinical decision support.