Quick Take
- The finding: Timely abstention — stopping at the first turn where abstaining is justified — averages under 40% for every model tested.
- They linger: On web shopping, the strongest baseline reaches only 26.7% timely abstention recall but 83.2% overall recall by turn 10.
- Scaffolding helps: With Llama-3.3-70B, a scaffold raises timely recall from 26.7% to 57.4% and overall recall from 83.2% to 100%.
Agents are built to act. University of Washington, University of Leeds, Southwest Jiaotong and Allen Institute for AI researchers ask the opposite skill: when should an agent stop? Agentic abstention is sequential — answer, abstain, or gather more information each turn — and often only becomes clear after interacting with the environment.
What the study shows
Evaluation covers web shopping, terminal environments and QA across 13 LLM-as-agent systems and 2 scaffolds on more than 28,000 tasks. Most systems average under 50% abstention recall.
- Terminal is also hard: Best timely recall on abstention tasks is only 21.6%.
- False premises stump everyone: On QA with false premises or underspecified intent, timely and overall recall stay at or below 50% for all models.
- Over-abstention rises with turns: Qwen3-235B-Instruct’s over-abstention reaches 34% by turn 10.
- Reasoning modulates terminal over-abstention: GPT-5.4-mini with low reasoning plateaus near 8% over-abstention; medium/high stay at 0–2%.
Why it matters: endless tool loops are expensive
An agent that keeps searching for a product that does not exist wastes money and user trust. Knowing when to stop is a safety and cost feature.
Method in brief
Defines timely vs overall abstention recall across environments where instructions look feasible until the environment reveals otherwise.
What remains open
- Environment design: Abstention labels depend on how impossibility is encoded.
- Trade-offs: Higher abstention can raise over-abstention on solvable tasks.
Context
Related agent reliability work is in Agents and Safety & Alignment.
What it means for agent designers
- Score timely stop: Overall abstention is not enough.
- Add abstain actions to scaffolds: They move the needle more than scale alone.
- Special-case false premises: QA abstention is especially weak.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “Agentic Abstention: Do Agents Know When to Stop Instead of Act?” (arXiv:2606.28733v1, submitted June 27, 2026). All numbers are taken from the paper’s text, tables and figures; i10X did not rerun the experiments. The paper is distributed under arXiv’s non-exclusive license, so i10X does not reproduce its figures; the chart was drawn by i10X from numbers reported in web environment results.
i10x Perspective
Most agent papers celebrate more tools and more turns. This one asks for fewer. The 26.7% timely recall figure should be on every agent scorecard next to task success.
