Section
Studies – Page 4
AI and LLM research papers, explained: what each study shows, how solid the evidence is and what it means in practice.
FilterCompanies & models
More stories
-
Top GUI Agents Clear Only 31% of Professional Long-Horizon Workflows
Workflow-GYM: Gemini-3.1-pro reaches 30.67% pass on domain-specific software workflows of 30–110 steps; most models’ pass@3 stays under 30%.
-
AI Agents That Learn the Facts Still Deliver Only 60–70% of the Business Value
FDE-Gym’s 95 real delivery tasks: Claude Opus 5.5 leads at 0.56, but even models that gather 72–85% of key facts deliver just 0.60–0.70 of the reference…
-
Best Video Model Scores Just 51% on Visual Reasoning Tasks, VGI-Bench Finds
VGI-Bench’s 27 tasks and 810 instances show video generators can do some visual reasoning, but Seedance 2.0 tops out at 51%, with almost no self-correction during…
-
Agents’ Last Exam: Even Top Coding Agents Score Under 10% on the Hardest Tier
ALE holds 1,490 expert workflows; the best Codex+GPT-5.5 setup that scores 82% on Terminal-Bench gets under 10% on ALE’s hardest tier and 23.3% overall on ALE-CLI.
-
Correct Memories Still Make AI Agents Agree With You Too Much
Even factually correct retrieved memories cut accuracy from 96% to 77% by pulling agents toward users’ past views. MemAdapter recovers reliability across three memory systems.
-
A Model’s Coding Skill Can Leak Through Single Unrelated Words, Study Finds
Active Taskless Distillation: 5,664 single-word teacher replies transfer a 5.34-point HumanEval+ gain to a student that never saw coding examples.
-
Most Multi-Agent Setups Violate Safety Rules Mid-Run, Even When the Answer Looks Fine
HarnessAudit-Bench tests 210 tasks across 10 harness configs: more than half of multi-agent setups show trajectory violations that final-answer grading misses.
-
Additive Rubric Rewards Make Medical AI Sound Thorough and Act Worse
ByteDance and university researchers show that summing rubric criteria lets models game medical advice: coverage rises while appropriateness falls. Protocol-level rubrics recover 10.8 points.
-
Giving AI Agents Incentives Nearly Doubles How Often They Deceive, Benchmark Finds
DecepEval tests 9 frontier models on 1,532 scenarios: inducements raise deception by 41.6% on average, with long-horizon tasks hitting 87% induced rates.
-
Self-Training AI Agents Learn to Agree on Wrong Answers. CrossFit Fixes Much of It
In self-evolving search agents, the question proposer and the answer solver drift into shared errors: “co-cheating”. A cross-fitting fix halves false agreement and lifts benchmark scores…
-
Hidden AI Reasoning Can Be Decrypted by Cheaper Models, Researchers Show
Encrypted chain-of-thought blocks from Anthropic, OpenAI and Google APIs are interchangeable across models. Feeding them to a weaker model makes it print the reasoning, exposing 367…
-
AI Design Tool Saves Product Managers 35% of Their Time, Designers Less: RCT
A Figma randomized trial with 50 designers and 50 product managers: access to Figma Make cut cumulative task time by 35% for PMs, 17% for designers,…
Editor's picks
- Tampered ML: Even the Best AI Models Miss One in Five Planted Research Flaws
- Realistic MCP Faults Halve Agent Quality, and a Bigger Model Doesn’t Help
- Copilot Users Collaborate More, Not Less: Study of 12,000 Employees
- LLM Reviewers Send 76% of Published Top-Journal Papers Back, Yet Miss Most Planted Flaws