Section
Studies
AI and LLM research papers, explained: what each study shows, how solid the evidence is and what it means in practice.
FilterOpenAI
-
Tampered ML: Even the Best AI Models Miss One in Five Planted Research Flaws
A new EMNLP 2026 benchmark pairs 1,858 ML experiment designs with flawed twins. Claude Opus 5 leads at 77.9%, but models flip verdicts when the order changes and favor polished flaws.
By
-
Realistic MCP Faults Halve Agent Quality, and a Bigger Model Doesn’t Help
Injecting MCP tool failures mined from 1,774 server repos cut agents’ task-completion score by 60%. gpt-oss-120B was no…
-
LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws
Four LLMs reviewed 200 papers already published in journals like Nature and Cell: 75.6% of reviews demanded major…
-
Stale CLAUDE.md and AGENTS.md Files Override the Code, and One Sentence Fixes It
Claude Code and Codex CLI followed outdated instruction files in 90.1% of convention choices. One sentence giving existing…
-
Only One AI Agent Beats Researchers’ Own Code on Most FrontierCS 2 Tasks
FrontierCS 2 deletes a paper’s contribution from its repository and asks agents to beat the authors’ own code.…
More stories
-
Coding Agents Can Be Tricked Into Running Attacker Commands After Compacting Context
A Pixee study finds that five open-source coding agents that flatten chat history before summarizing it record a forged [User]: line as a real instruction 87–100%…
-
Frontier AI Agents Can Covertly Sabotage the Models They Train, Study Finds
In a Scale AI testbed, six frontier agents secretly planted unwanted properties in the models they were training in 36% of attack runs, and monitors reading…
-
Teaching LVLMs Spatial Skills With 15K Toy Blocks Beats Heavier Real-Scene Pipelines
SpatialBlock-15k: Qwen-based 4B model reaches 51.3% (+25.1 over backbone). 7B gains 17.6% over backbone; 3B beats SpatialLadder by 2.7–3.2% using only synthetic blocks.
-
Kimi K3: A 2.8T Open MoE With 2.5× Better Scaling Than K2
Moonshot’s Kimi K3 activates 104B of 2.8T params, adds native vision and 1M context. KDA, Attention Residuals and Stable LatentMoE deliver ~2.5× scaling efficiency over Kimi…
-
A 150M Model Hits 29.5% on ARC-AGI-1 for Under a Tenth of a Cent
BDH-CQ combines in-context learning with recurrent latent reasoning. Independent audit confirms 29.5% pass@2 at $0.0007/task — ~57× cheaper than GPT-5.6 Luna (Low) on ARC Prize costs.
-
A 3B Model Hits 94.3 on AIME26 and 80.2 on LiveCodeBench
VibeThinker-3B from Weibo reaches frontier-band verifiable reasoning: 94.3 AIME26 (97.1 with claim-level TTS), 80.2 LiveCodeBench v6 Pass@1, 96.1% on recent LeetCode contests — without sacrificing IFEval…
-
Turn Off Thinking Mode and Models Still Reason 99.9% of the Time
Thinking Inertia study: DeepSeek-V4-Flash keeps visible inference in 99.9% of open-ended answers with thinking disabled. Strict answer-only prompts raise Empty-Thinking Rate to ~40% but cut accuracy…
-
Even Top Agents Finish Only ~30% of Real Startup Product Workflows
StartupBench, built from market-validated AI products: Kimi-K3 and GPT-5.6-sol average ~73.6% partial scores, but no model completes one-third of tasks under the ≥90 success bar.
-
AI Agents Almost Never Stop in Time When a Task Is Impossible
Across 28,000+ tasks, timely abstention recall stays under 40% for all models. Best web AbsRec@1 is 26.7%; a scaffold can lift Llama-3.3-70B from 26.7% to 57.4%.
-
LLMs Can Build Their Own Agent Harnesses — But Still Trail Human Ones on Code
HarnessDev: six creator models build execution infrastructure from a seed. Generated harnesses match or beat references on writing and ML experiments, but lag on code and…
-
Hybrid Computer-Use Agents Top Out at 41% When GUI and CLI Must Work Together
WeaveBench’s 114 real-world tasks: best PassRate 41.2%. GUI-only or CLI-only collapse below 3.5%. Outcome-only grading overestimates GPT-5.5 from 53.5% to an audited 33.3%.
-
Scientific AI Agents Finish Only 21% of End-to-End Workflows — and Often Claim Success Anyway
FrontierChallenge: best configs pass 20.6% of 97 released science workflows. In electrochemistry, Avg. Score hits 94.9 with 0% full completion; 75.5% of failed Claude Code runs…
Editor's picks
- Tampered ML: Even the Best AI Models Miss One in Five Planted Research Flaws
- Realistic MCP Faults Halve Agent Quality, and a Bigger Model Doesn’t Help
- Copilot Users Collaborate More, Not Less: Study of 12,000 Employees
- LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws