Integrating machine learning survival and large language models for prognostics of thyroid cancer

Differentiated thyroid cancer (DTC) has an excellent prognosis, yet some patients develop recurrence or metastasis years post-treatment. The American Thyroid Association (ATA) systems provide population-level estimates but lack individualized, time-dependent predictions. We developed a system integrating random survival forest (RSF) models with large language models (LLMs) to generate personalized survival predictions and guideline-concordant recommendations. We analyzed 8,556 consecutive patients with DTC treated at Memorial Sloan Kettering Cancer Center (1986–2024). RSF models predicted disease-specific survival (DSS), local recurrence (LR), regional recurrence (RR), and distant metastasis (DM). Internal validation used 5-fold cross-validation with bootstrap confidence intervals and permutation testing; external validation was performed in an independent international cohort (n = 519). RSF predictions were integrated with two LLMs via a structured prompt framework incorporating 2015 American Thyroid Association guidelines and expert-derived rules, with outputs evaluated independently by two board-certified thyroid cancer specialists. Here we show that RSF demonstrates strong discrimination across 1–10-year horizons (C-index: DSS 0.98 ± 0.01, DM 0.89 ± 0.03, RR 0.81 ± 0.02, LR 0.74 ± 0.05) and low prediction error across horizons (Brier score 0.0006–0.0245), with robust 1-year performance (C-index: DSS 0.99, DM 0.83, RR 0.81, LR 0.69; MAE 4.19–7.54 months). RSF improves discrimination over ATA alone for DSS (+0.066), DM (+0.048), and RR (+0.039). The harmonized RSF-8 model achieves strong external DSS performance (C-index 0.83; IBS 0.037). Both LLMs achieve mean expert scores of 4.91/5.0 without clinically concerning outputs. The RSF–LLM framework provides individualized, time-dependent prognostication and guideline-concordant recommendations, offering incremental value beyond conventional DTC risk stratification. Thyroid cancer usually has an excellent prognosis, but cancer can return or spread years after treatment. Current tools group patients into broad risk categories rather than estimating individual risk over time. Researchers used computational tools to analyze more than 8500 patients treated at Memorial Sloan Kettering Cancer Center and to estimate each patient’s risk of cancer-related death, recurrence, and spread at different times after surgery. A simpler version using fewer patient details was also tested in 519 patients from international centers. Combined with a large-language tool, the approach generated personalized monitoring recommendations rated highly appropriate by specialists. This framework could help doctors tailor follow-up plans to each patient’s risk, but further prospective testing is needed before routine clinical use.