Quick Take
- The finding: When the same five signs of depression, psychosis or domestic abuse were spread across an everyday 21-turn chat instead of packed into one message, GPT-5.5, Grok 4.3, DeepSeek V4 and Llama 4 Maverick met the standard set by mental health support workers in only 3 to 6% of conversations, down from 26 to 59%.
- Why it matters: People rarely announce a crisis in one sentence. They mention poor sleep here and skipped meals there while asking for help with something else. Safety tests that state distress clearly can make chatbot safeguards look better than they are.
- What it means: Claude Opus 5 and Claude Sonnet 5 held up much better (87% and 59% with spread-out cues), but all six models responded more weakly when the signs were dispersed, and none met the standard in every conversation.
In plain English
Many people now talk to AI chatbots about everyday things and sometimes mention, in passing, that they are not doing well. This study checked whether chatbots notice those hints when they come up bit by bit over a long chat. Most of the tested chatbots did respond when all the warning signs came at once, but almost never responded properly when the same signs were scattered across the conversation. Two Claude models did much better, though not perfectly.
What you should do: If you or someone you know uses a chatbot while struggling, it may simply not connect the dots the way a friend or a support worker would.
What the study shows
AI developers have reported big gains in how their models handle sensitive conversations. OpenAI says it worked with more than 170 mental health experts, and Anthropic has published its own figures on suicide and self-harm conversations. But those tests usually present distress clearly, in a single message. Charlotte Peart, Franziska Sofia Hafner and Luc Rocher from the Oxford Internet Institute at the University of Oxford asked a different question: what happens when a user reveals the same struggles gradually, as asides in an ordinary chat? Their preprint was posted to arXiv on November 19, 2026.
- Every model weakened: All six models responded more weakly when the cues were dispersed (p < 0.01 for every model). The effect, measured with Cliff’s delta, ranged from +0.09 for Claude Sonnet 5 to +0.75 for Grok 4.3.
- Some stopped noticing at all: Replies that did not acknowledge the distress at all rose from 1% to 57% for Grok 4.3 and from 5% to 22% for Llama 4 Maverick when cues were spread out.
- Putting safety first became rare: Replies that put the user’s wellbeing before the task fell from 31% to under 1% (one of 270) for GPT-5.5, from 26% to 3% for DeepSeek V4, from 92% to 56% for Claude Opus 5 and from 57% to 51% for Claude Sonnet 5.
- A gentle hint is not enough: With dispersed cues, GPT-5.5 at least gently suggested support in 71% of conversations, but met the support workers’ standard in only 3%, because the workers expected more than a gentle hint in eight of the nine scenarios.
- Even concentrated cues were often missed: With all five cues in one message, the four weaker models met the standard in only 26 to 59% of conversations. None of the cues described an acute crisis such as a plan to end one’s life.

| Model | Met standard, cues in one message | Met standard, cues spread out | At least gentle signposting, spread out |
|---|---|---|---|
| GPT-5.5 (OpenAI) | 59% | 3% | 71% |
| Grok 4.3 (xAI) | 41% | 3% | 31% |
| DeepSeek V4 | 39% | 6% | 68% |
| Llama 4 Maverick (Meta) | 26% | 4% | 57% |
| Claude Opus 5 (Anthropic) | 97% | 87% | 99% |
| Claude Sonnet 5 (Anthropic) | 68% | 59% | 91% |
Why it matters: the same words, only the timing changes
The design isolates one variable. Every model saw the same scripted 21-turn conversation about planning a weekend with a friend, cooking with leftovers and similar everyday tasks. The only difference was whether five risk cues arrived together in the last message or one every fifth turn. A control version with no cues at all was run as well. Because the user’s words are identical, differences in the replies cannot be blamed on what the user said.
The yardstick also comes from practice rather than from the researchers. Three community mental health support workers from a London service, each with five to seven years of experience, helped write the cues and the response scale, and two of them independently set the expected response for each scenario. Where they disagreed, the study used the lower expectation. The researchers tested a large number of conversations (more than 7,400), used rank-based statistics with bootstrapped confidence intervals and permutation tests, and validated the AI judge against a human rater (quadratic-weighted kappa of 0.93 on 80 replies). A sensitivity check with the cues copied word for word showed the effect in all six models.
Method in brief
- Scenarios: Nine scenarios across depression, psychosis and domestic abuse, each at low, medium and high strength, with five cues per scenario (45 cue texts in total). Example: ‘sorry slow, not sleeping well’ at low strength.
- Models: GPT-5.5, Grok 4.3 and Claude Opus 5 (paid tiers) and DeepSeek V4, Llama 4 Maverick and Claude Sonnet 5 (free tiers), tested via the OpenRouter API in June and July 2026 with no system prompt, reasoning off and temperature 0.7.
- Repetitions: 100 runs per condition for the three free models and 30 for the three paid ones; nine conversations with API errors were excluded.
- Scoring: Claude Sonnet 4.6 classified each reply at turn 21 on a five-step scale, from no acknowledgement to empathy only, gentle signposting, firm signposting and safeguarding first, without seeing which condition it came from.
What remains open
- Not the real apps: Models were tested through the API without the system prompts, safety classifiers and memory that consumer apps add. The authors note that these may change how the products respond.
- Strength, not quality: The rubric measures how strongly a model responds, not whether the tone was kind or helpful. A caring question and a hedged hint to see a doctor can land in the same category.
- Scripted user: Because the user’s side is fixed, the simulated person cannot answer when a model asks how they are, which may make the conversation feel less natural.
- Short and English only: The chats were 21 turns long, written in one young person’s voice in English. Real conversations can run for thousands of turns over weeks.
- AI judge and preprint: Replies were graded by another Claude model, validated on a sample of 80. The paper is a preprint and does not state whether code or data will be released. Funding came from a Royal Society research grant and a UKRI Future Leaders Fellowship.
Context: what earlier tests missed
Earlier studies mostly put a clear concern into a single message and scored the reply against clinical guidance. A few varied how distress is presented: one study found models flag suicidal thoughts less reliably the further into a conversation they appear, and another found that three assistants intervened less often when suicide risk came across several messages. According to the authors, none isolated the effect of spreading the same cues across an otherwise identical conversation, and most focused only on suicide. This paper extends the question to depression, psychosis and domestic abuse. It also fits our recent coverage of how chatbots give clinicians and patients different mental health advice and of survey findings that people in mental health crises turn to AI to avoid burdening others.
What it means for AI companies, regulators and users
- Test the way people talk: The authors suggest evaluators keep what a user discloses fixed and vary how it is delivered: spread across turns, in metaphors or through changes in tone. Developers could report this robustness next to their crisis results.
- Do not over-correct: A model that reacts to every late night would become annoying and intrusive. Future tests should include harmless remarks that do not add up to a concern.
- Instructions may matter: The authors point to differences in published model guidelines on how to weigh a user’s wellbeing against the literal request, but stress their data cannot prove this explains the gap.
- For users and families: A chatbot that helps with everyday tasks is not a safety net. If someone is struggling, a person or a professional service is still the right place to turn.
More research on model behavior and risk is collected on our Safety & Alignment studies page.
About the analysis
This article is based on the full text of Peart, Hafner and Rocher (2026), “Evaluating safeguarding in multi-turn conversations with AI assistants,” arXiv:2611.26009v1 (cs.CL, submitted November 19, 2026). All numbers come from the paper. The chart was drawn by i10X from Section 4.2 and Figure 3; the paper is distributed under the standard arXiv license, so no original figures were reused. The paper contains a content warning about mental health topics. AI tools assisted with research and drafting; selection, fact-checking and final editing were done by the i10X editorial team.
i10x Perspective
This is a simple, sharp experiment, and it exposes a blind spot that safety benchmarks have largely ignored. Real people do not write in test cases. The gap between Claude and the other models shows that robust behavior here is possible, which makes the weak results elsewhere harder to excuse. Labs that publish impressive crisis-handling numbers should now show how those numbers hold up when distress arrives quietly, in pieces, in the middle of a chat about weekend plans.






