Structured Workflows and Retrieval-Augmented Generation Improve LLM Recommendations for Colorectal Liver Metastases: The METATRON Study

Abstract Large language models (LLMs) show promise for clinical decision support, but their ability to reproduce multidisciplinary oncologic reasoning is unclear. We evaluated whether structured clinical workflows and retrieval-augmented generation (RAG) improve concordance between LLM-generated and expert reference recommendations across 110 synthetic colorectal liver metastasis (CRLM) vignettes spanning eight clinical scenarios. Four inference strategies built on the same LLM (gpt-oss-120B)—baseline prompting, RAG, workflow-guided prompting, and their combination (WF-RAG-LLM)—were scored by three independent LLM judges across five dimensions. WF-RAG-LLM achieved the highest aggregated score (0.626 ± 0.070), ahead of RAG-LLM (0.559 ± 0.111), WF-LLM (0.527 ± 0.168), and baseline (0.444 ± 0.162), with the largest gains in post-chemotherapy, recurrent, and anatomically challenging scenarios. Combining expert-derived workflow guidance with retrieval-based knowledge grounding achieved the highest overall concordance with expert reference recommendations, supporting clinically informed reasoning architecture as an important component of LLM performance in complex oncologic decision support.