Concordance between multidisciplinary tumor board decisions and large language model recommendations in gynecologic oncology: a retrospective study

Multidisciplinary tumor boards (MDTBs) play a central role in developing patient-centered treatment strategies for gynecologic malignancies. In recent years, large language models (LLMs) have emerged as potential clinical decision-support tools. This study aimed to evaluate the concordance between gynecologic oncology MDTB decisions and treatment recommendations generated by two different LLMs (ChatGPT-5.2 and Gemini Pro), while also exploring the potential limitations and safety-related considerations of these models in oncologic decision-making. In this retrospective observational study, 114 patients evaluated by a multidisciplinary gynecologic oncology tumor board between June and December 2025 were included. Patients were included consecutively from all eligible cases evaluated during the study period. Standardized case summaries presented during MDTB meetings were submitted to ChatGPT-5.2 and Gemini Pro using predefined system prompts incorporating specified guideline versions and knowledge cutoff dates. The recommendations generated by the LLMs were compared with final MDTB decisions. Concordance was assessed using Cohen’s kappa coefficient, and paired comparisons between the models were performed using the McNemar test. The concordance rate between ChatGPT-5.2 recommendations and MDTB decisions was 78.9% ( n = 90; κ = 0.729), whereas the concordance rate between Gemini Pro recommendations and MDTB decisions was 76.3% ( n = 87; κ = 0.694). In diagnosis-specific analyses, ChatGPT-5.2 demonstrated the highest concordance in tubo-ovarian cancers (κ = 0.769), while Gemini Pro showed the highest concordance in endometrial cancer (κ = 0.759). In cervical cancer, the κ values were 0.653 for ChatGPT-5.2 and 0.595 for Gemini Pro. No statistically significant difference was observed between the models regarding concordance with MDTB decisions (McNemar p = 0.607). In exploratory surgical recommendation analyses, ChatGPT-5.2 showed higher specificity (97.7%), whereas Gemini Pro showed higher sensitivity (74.1%). Large language models demonstrated substantial concordance with multidisciplinary tumor board decisions in gynecologic oncology and may serve as supportive clinical decision-support tools. However, concordance with MDTB decisions does not indicate clinical accuracy or patient safety. Limitations related to text-based assessment, incomplete representation of patient-specific clinical factors, and potential discordance in individualized treatment scenarios highlight the need for expert physician oversight. Therefore, LLMs should be considered complementary tools that support, rather than replace, multidisciplinary clinical decision-making.