Abstract / Summary
Conversational artificial intelligence is increasingly encountered by people seeking support during distress, social isolation, or delays in accessing mental-health care. In these settings, the central ethical concern extends beyond factual inaccuracy. Large language models (LLMs) may produce responses that appear empathic while failing to detect suicidal intent, exceeding the limits of a non-clinician support role, undermining patient autonomy, or weakening the clinician–patient relationship. This narrative review reframes ethical boundaries for LLMs in mental-health care around three clinically important contexts: crisis support, psychiatric triage, and therapy augmentation. A narrative approach was adopted to support conceptual synthesis rather than pooled effect estimation. The literature on digital psychiatry, conversational agents, LLMs, artificial intelligence ethics, suicide-safety responses, patient perspectives, and clinical reporting guidance was reviewed from major medical and interdisciplinary sources. The primary search covered publications from 1 January 2021 to 30 June 2026, with earlier foundational chatbot studies retained selectively and interpreted separately from contemporary LLM evidence. Four interrelated priorities are proposed for evaluation: governance centred on psychological safety, autonomy, trust and the clinician–patient relationship; crisis-response safeguards combining supportive recognition with human escalation; bounded lower-acuity support with transparent uncertainty; and co-designed evaluation of patient-experienced and relational outcomes. These are candidate design priorities, not demonstrated mechanisms of risk reduction. The proposed framework integrates existing ethical principles with mental-health scenarios, external control functions, verified human handoffs, and staged evaluation. Study-level findings are distinguished from technical background and author-derived proposals. The framework is exploratory and has not been empirically or clinically validated; it is intended to generate research questions, not guide routine clinical care. Future studies would need to assess clinically meaningful outcomes: whether the system recognises acute risk, maintains a bounded support role, preserves patient autonomy and trust, and enables timely human escalation when vulnerability, ambiguity, or risk is disclosed.