Quick Take
- The finding: At the same token budget, a coding agent resolved more real GitHub issues when its context held a few whole, unedited files than when the same budget went to prompt compressors or code retrievers: 43.7% of 300 issues with GPT-5.6 and 46.7% with Claude Opus 5.
- Why compression hurts: Retrievers often never show the file that has to change, and token pruners keep that file but alter its text, so the model writes patches against lines that no longer exist and those patches fail to apply.
- What it means: For teams building on OpenAI or Claude models, the cheapest win is choosing which files the agent sees, then showing them verbatim. The selector was also the cheaper pipeline: $0.151 per issue with GPT-5.6, against $0.168–0.177 for the compressor pipelines.
Every coding agent hits the same wall: real repositories are far bigger than any context window. The codebases in SWE-bench average 3,010 non-test files and 438,000 lines, so an agent can only ever look at a small slice. The industry’s standard answers are prompt compressors, which delete “unimportant” tokens, and retrievers, which rank code chunks by how similar they look to the bug report. A new preprint from the University of Illinois Urbana-Champaign tests both against a much plainer idea, and the plain idea wins.
What the study shows
The authors built a simple selector called CQ2-CARE. It asks the model twice which files must change to fix an issue, keeps only the files both answers name, and shows them whole and exactly as they are in the repository, plus a few short spans that grep finds for definitions, callers or strings the model asks for. Every competing method then got exactly the same per-issue token budget.
| Context method | Resolved, GPT-5.6 | Resolved, Claude Opus 5 | Patches that apply (GPT-5.6) |
|---|---|---|---|
| CQ2-CARE (whole files) | 43.7% | 46.7% | 98.0% |
| SweRank (trained retriever) | 43.0% | 45.0% | 97.3% |
| SWE-Pruner (trained line pruner) | 41.7% | 39.3% | 93.0% |
| LongCodeZip (compressor) | 37.3% | 36.0% | 89.7% |
| LLMLingua-2 (compressor) | 30.0% | 40.3% | 70.7% |
| BM25, whole files (retriever) | 34.3% | 34.0% | 95.7% |
| Dense, chunks (retriever) | 22.7% | 33.7% | 78.3% |
| Gold files (upper bound) | 49.3% | 46.3% | 97.7% |
- Beats every drop-in baseline: CQ2-CARE resolved 6.3 points more issues than the best of three prompt compressors with either model, and 9.3 (GPT-5.6) and 12.7 (Claude Opus 5) points more than the best of three retrievers.
- Edges out trained methods: It also resolved more than two systems trained specifically for issue resolution: SweRank by 0.7 and 1.7 points, and SWE-Pruner by 2.0 and 7.3 points. The authors stress that the small gap to SweRank does not show the two are equal.
- Close to knowing the answer: Showing the true files to edit (the “gold” oracle) resolved only 5.7 points more than CQ2-CARE with GPT-5.6, and no more with Claude.
- Altered text breaks patches: LLMLingua-2 kept just 17.5% of context lines verbatim, and only 70.7% of its patches applied, against 98.0% for CQ2-CARE. In a controlled test, deleting 20% of the tokens from every line of CQ2-CARE’s own contexts cut patch applicability from 98.0% to 78.7% and resolution by 9.7 points.
- Retrievers miss the target: Chunk-level BM25 and dense retrieval put a fifth or less of their context lines into the files that needed editing (CQ2-CARE: about half) and showed none of those files on more than a quarter of issues.
Why it matters: a careful comparison
Most context-compression papers grade their methods on question answering or summaries, where a few deleted words cost little. This study measures what actually matters for coding agents: whether the patch passes the project’s own tests. Several design choices make the result more convincing than a typical leaderboard claim:
- Matched budgets: every compressor and retriever was fitted to the same token count per issue, so no method wins simply by seeing more code.
- Test-graded, not exit-code graded: patches that fail to parse or apply count as failures. The authors note that exit-code grading, which they used before this study, credited an empty patch on 4 of 6 issues.
- Two model families and statistics: results hold for GPT-5.6 and Claude Opus 5, with paired McNemar tests and Holm correction; all main leads are significant, the narrowest (LLMLingua-2 with Claude, p = 0.027) only weakly.
- Controlled tests: the authors change one factor at a time (which files, how much text is altered), so they can show that each failure mode costs resolutions on its own.
Method in brief
Data: 300 issues from the SWE-rebench leaderboard, one per repository, created between January 2025 and May 2026. A salted hash drew 340 candidates before any model call; 328 passed an eligibility check (the gold patch fixes the issue and an empty patch does not), and the first 300 formed the test set. Models: gpt-5.6-sol at medium reasoning effort and Claude Opus 5, each used for every call. Baselines: three prompt compressors (LLMLingua-2, LongLLMLingua, LongCodeZip), three retrievers (chunk BM25, dense retrieval with CodeRankEmbed, and SWE-bench’s file-level BM25) and two trained systems (SWE-Pruner, SweRank). Task: the model reads the issue and one context, writes one unified diff, and the benchmark harness runs the tests.
With GPT-5.6, a CQ2-CARE context held 1.7 selected files on average (median 1) plus evidence spans from 6.0 more files, at a mean of 13.8K tokens (27.1K with Claude). An ablation found that the double-asking trick is not what makes it work: agreement between two answers raised file-localization F1 from 0.430 to 0.613, but mostly because it names half as many files; a single answer cut to the same number of files reached 0.600 and resolved about as many issues. The evidence step added 3.0 points, short of significance (p = 0.068).
What remains open
- Not a full agent loop: each method gets one patch attempt on Python issues from one benchmark. Real agents can re-open files or run tests, which may recover some of the losses; the authors leave that to future work.
- The budget is the selector’s own: every baseline is compared at CQ2-CARE’s operating point (a 100-issue sweep covers other budgets). Given CQ2-CARE’s own candidate files, LongCodeZip matched it (44.7% vs. 43.7%), so the lead in the main comparison comes from which files are kept.
- Noise and post-hoc additions: a second patch sample flips 8.7–18.9% of outcomes, so small gaps are fragile. File-level BM25, SWE-Pruner, SweRank and the controlled tests were added after the main results, and the written analysis plan was not time-stamped.
- Contamination: the issues may overlap the models’ training data, and SweRank may have trained on other pull requests from the same repositories, though the gaps do not shrink detectably on newer issues.
- Reproducibility: the authors say they will release code, prompts and per-issue outcomes; at the time of writing no repository is linked. The work has not been independently replicated.
Context: how it fits prior work
The result echoes earlier long-context research: Jha et al. (2024) found that extractive, reranker-based chunk selection often beats token pruning on question answering and summarization, and “lost in the middle” work showed models use information in the middle of long inputs less reliably. What this paper adds is the coding-specific failure: an edit must quote the exact code it changes, which is precisely what token pruning destroys. It also lines up with how agent tools already work. Search-and-replace editing in coding tools and Anthropic’s own SWE-bench write-up rely on verbatim quotes. LongCodeZip’s authors had warned that token-level compression “corrupts code syntax and structure”.
What it means for teams building agents
- Spend tokens on file choice, not compression: ask the model which files need to change, then show those files whole. The authors’ rule of thumb: choose the candidate files well, then cut whole lines if anything must be cut.
- Never feed edited code into an editor: if a pipeline must shorten code, it should drop whole lines or blocks, never individual tokens, or patches will reference text that does not exist.
- It is cheaper too: CQ2-CARE needs no compression model. It used 0.68× the tokens of the uncompressed context with GPT-5.6 (0.73× with Claude), while LongLLMLingua and LongCodeZip spent a median 34–37 seconds of GPU time per issue.
- Evaluate end to end: intermediate metrics such as localization scores can mislead. Compare context strategies at equal budgets with the benchmark’s real tests, as this study does.
Want to see how different models handle your own repository? Explore the latest agent coverage or the Anthropic and OpenAI hubs for the models used in this study.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “Skip the Compressor: Coding Agents Fix More Issues When They Pick Whole Files” (arXiv:2610.21073v1, submitted October 15, 2026). All numbers are taken from the paper’s text, tables and figures; i10X did not rerun the experiments. The charts were redrawn by i10X from the paper’s Table 1, which is published under a CC BY 4.0 license.
i10x Perspective
Context engineering has quietly become one of the most expensive parts of running coding agents, and the market has filled with clever compressors. This study is a useful corrective: for code, fidelity beats density. The winning recipe here is almost embarrassingly simple: ask the model where the fix lives and show it the real file. Expect agent frameworks to move toward file-level selection and line-preserving cuts. The open question is how much of this advantage survives inside multi-turn agents that can read, test and retry on their own.
