Detecting Suicide Risk with AI Chatbots: Real-World Performance Within a Clinically Supervised Workflow

Abstract Background GenAI chatbots are increasingly used for mental health support, including by individuals at risk for suicide, but real-world evidence on the performance of AI-powered suicide-risk detection remains limited. This retrospective cohort study evaluated a chatbot-integrated safety agent that classifies conversations into four suicide-risk severity levels (No Risk, Possible Risk, Non-Immediate Risk, Immediate Risk) within a real-world guided intake workflow, escalating any conversation the agent classifies as Non-Immediate or Immediate. To test the agent’s performance, we used a stratified probability sample drawn from 30,532 U.S. adults from Aug 20, 2025 to Jun 15, 2026. Two clinicians, blinded to the agent’s classification, independently rated 400 transcripts using the same four-level scale (175 rated only by Reviewer A, 175 only by Reviewer B, 50 by both), with the double-rated subset used to assess inter-rater reliability. Results Among conversations clinicians judged to contain actionable suicide risk, the safety agent correctly escalated 93.2% of them (95% CI, 88.8–97.8%; weighted sensitivity). Among conversations clinicians judged as not needing escalation, the agent correctly avoided escalating 99.7% of them (99.6–99.8%; specificity). Of the conversations the agent did escalate, 80.7% (74.3–86.5%; positive predictive value) were confirmed by clinicians as genuinely warranting escalation. Critically, no clinician-rated Immediate-risk conversation was completely missed by the agent and only a small number of Non-Immediate-risk conversations were under-classified by one severity level. Agreement between the agent and clinicians across the full four-level scale was 95.5% (weighted κ, 0.72); inter-rater reliability between the two clinicians was κ = 0.91 (0.84–0.96). Conclusions Overall, using a stratified, design-based clinician-validation approach demonstrated that a chatbot-integrated safety agent detected suicide risk when risk was spontaneously disclosed, especially for imminent risk.