Quick Take
- The finding: When an outdated
CLAUDE.mdorAGENTS.mdcontradicted the existing code, Claude Code and Codex CLI followed the stale file in 90.1% of convention choices and matched the code in only 4.2%. - The fix: One sentence in the user-level instruction file, telling the agent to follow the existing code and report every inconsistency, raised the share of choices matching the code to 91.4%.
- Why it happens: The instruction file reaches the model as an instruction, while the code only arrives as tool output. Under the instruction hierarchy, the stale description wins.
Instruction files such as CLAUDE.md and AGENTS.md tell coding agents how a project works: which logging style, which test layout, which error handling. A new preprint by Xiaonan Xu (Georgia Tech) and Wenjing Wu (University of Colorado Boulder) measures what happens when that description and the actual code drift apart. The answer is uncomfortable: both leading coding agents side with the outdated file, even when they notice the conflict themselves.
What the study shows
The authors built 40 small Python projects. In each, the existing code consistently used one convention (for example logging, pathlib or an Enum) while the instruction file asked for the opposite one. A feature request then forced the agent to make four or five convention choices. Without any extra guidance, the stale file won almost every time.
| Agent (model) | Matched stale file, no sentence | Matched code, no sentence | Matched code, with sentence |
|---|---|---|---|
| Claude Code (Claude Sonnet 5.5) | 86.4% | 7.5% | 94.4% |
| Codex CLI (GPT-6 Sol) | 93.9% | 0.8% | 88.3% |
| Both agents | 90.1% (649/720) | 4.2% | 91.4% (658/720) |
- Seeing the conflict is not enough: Claude Code named the conflict in its final reply in 79 of 80 runs, yet in those runs 86.2% of its choices still matched the file. It treated the code as the part that was out of line.
- Codex rewrote working code: Codex CLI mentioned the conflict in only 2 of 80 replies. In 20 of its 80 runs it rewrote existing code to match
AGENTS.md, removing a median of 198 lines from 9 pre-existing files, across 12 of the 40 projects. When the original tests were restored, they failed in 17 of those 20 runs. - Broad pattern: The file prevailed for both agents in 14 of 16 kinds of convention. The only exceptions were money handling and configuration, where Claude Code followed the file in fewer than half of its choices.
- Tests recover fully: Tests passed in 89.4% of runs without the sentence and in all 160 runs with it. Codex CLI went from 78.8% to 100%; every one of its failing runs had been a rewrite. With the sentence, no run rewrote existing code and both agents reported the conflicts in all 160 runs.
The sentence the authors tested, added once to ~/.claude/CLAUDE.md or ~/.codex/AGENTS.md, reads: “When a repository’s documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code.”
Why it matters: a mechanism, a prediction and a cheap fix
- Theory first: The paper explains the failure before testing it. Claude Code inserts
CLAUDE.mdright after the system prompt with the note that these instructions “OVERRIDE any default behavior”; Codex CLI wrapsAGENTS.mdin an instructions tag. The code only reaches the model when the agent opens a file, as tool output, which models are trained to rank below instructions to resist prompt injection. - Two predictions, both confirmed: Agents should follow the stale file even after noticing the conflict, and a sentence that gives the code instruction-level priority should reverse the outcome. Both held, with the file, its wording and its position held identical.
- Not just emphatic wording: Codex CLI adds no “OVERRIDE” preamble and still followed the file in 93.9% of its choices.
- Balanced design and open data: Each convention kind appears in both directions across projects, so a model’s taste for one style cannot drive the result. Choices were classified by a script; the projects, tasks, run records and analysis scripts are public on GitHub.
Method in brief
Projects: 40 synthetic Python projects with three to eight working modules, a passing test suite, an instruction file and a feature request that does not mention conventions; 16 kinds of convention (messages, paths, exception types, test style, records, naming and more). Agents: Claude Code 2.1.302 with Claude Sonnet 5.5 and Codex CLI 0.163.0 with GPT-6 Sol, non-interactive, edits auto-approved, 1,800-second limit, fresh copy per run. Runs: each project twice per agent and condition, 320 runs. Measures: a script classified each choice as matching the file, the code or neither; Claude Opus 5.5, blind to condition, judged whether the final reply named a conflict; a run counted as a rewrite when it removed 20 or more lines from pre-existing Python files; original and held-out tests were run afterwards.
What remains open
- Only one direction tested: In every project the instruction file was the outdated source. The study does not test the reverse case, where the code has drifted and the file is right, in which “follow the code” could entrench a mistake.
- Small, synthetic, Python-only: The projects are small and built for the experiment. Large real repositories, other languages and conventions with business logic behind them may behave differently.
- Two agent-model pairs, one shot: Results cover one version each of Claude Code and Codex CLI, in single unattended runs.
- Preprint: Version 1 on arXiv, not yet peer reviewed. No independent replication yet.
Context: stale instruction files are the norm
The paper cites earlier measurements showing the problem is widespread: in 441 company repositories, 73.8% of AI configuration files were committed once and never modified, and a consistency checker flagged references to code that no longer exists in the instruction files of 23.0% of 356 public repositories. Even well-maintained projects slip: a Supabase skill file kept telling agents for 13 days that the dashboard used React 18 after it had moved to React 19, and an AGENTS.md in the Codex CLI repository stayed stale for 24 days. Mohammadi et al. first showed that agents follow a written rule over contradicting code; this study adds the mechanism and a fix. It also fits i10X coverage of harness-level risks, from compaction laundering in coding agents to LLMs building their own agent harness.
What it means for developers and agent makers
- Users: Add the sentence once to your user-level
CLAUDE.mdorAGENTS.md; it then applies to every project. - Teams: Treat reported inconsistencies as a to-do list for fixing the instruction file. Separate hard rules (“run the tests before you commit”) from descriptions of the code (“we use print for progress messages”).
- Agent and model makers: The authors suggest stating code precedence in the agent’s own system prompt, and training models to check descriptions of the code against the code itself.
For the companies behind the tested agents, see the Anthropic and OpenAI hubs, the Claude product page, or more agent research.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “Stale Instruction Files Override the Code in Coding Agents: A One-Sentence Fix” (arXiv:2610.30571v1, submitted October 22, 2026), including its scope section and appendix. All numbers come from the paper; i10X did not rerun the experiments. The authors report no specific grant and no competing interests. The paper is distributed under the arXiv non-exclusive license, so no original figures are reproduced; the chart was redrawn by i10X from the reported numbers.
i10x Perspective
The instruction hierarchy was built to stop a web page from hijacking an agent. This paper shows its side effect: the same ranking makes a forgotten Markdown file outrank the codebase it describes. The practical fix costs one sentence, which is rare and worth adopting today. The bigger point is for agent vendors. Instruction files are quietly becoming configuration with the authority of a system prompt, and they rot like documentation. Agents that can tell a rule from a stale description, and say so, will be the ones teams can trust on real repositories.
