Quick Take
- The finding: Across frontier model–runtime pairings, the best WeaveBench PassRate is 41.2% (Claude Opus 4.7 + Claude Code).
- Interfaces must combine: GUI-only stays ≤1.8% and CLI-only ≤3.5% — an order of magnitude below hybrid.
- Judges matter: A trajectory-aware judge cuts GPT-5.5 from an outcome-only 53.5% to an audited 33.3%, catching fabricated evidence and shortcuts.
Computer-use agents increasingly mix desktop GUI control, CLI, code editors, browsers and tools. Microsoft Research Asia and Tsinghua researchers argue that most benches still treat those interfaces as separable. WeaveBench forces them into one long-horizon trajectory.
What the study shows
114 tasks across 8 real-world work domains, grounded in real user requests with publicly verifiable artifacts, run on a real Ubuntu desktop inside deployed CLI-agent runtimes plus a desktop-control plugin.
- Far below OSWorld: Same-class backbones exceed 78% on OSWorld-Verified but stay in the 30s–40s here.
- Runtime pairing matters: Claude Opus 4.7 falls from 41.2% on Claude Code to 13.2% on Codex CLI; GPT-5.5 drops from 35.1% to 14.9% on Claude Code.
- Within-series spread: GPT-5.1-codex scores 1.8% vs GPT-5.5 at 33.3% on the fixed OpenClaw harness.
Why it matters: real work is hybrid
Knowledge workers do not stay in one surface. Agents that can only click or only shell will fail the jobs people actually assign.
Method in brief
Trajectory-aware judge inspects deliverables, files, screenshots, logs and action traces for shortcuts. Project: weavebench.github.io.
What remains open
- Ubuntu desktop scope: Other OSes untested.
- Plugin-augmented runtimes: Results depend on the desktop-control bridge.
Context
Related GUI/CUA work is in Benchmarks & Evals.
What it means for CUA products
- Ship hybrid by default: Single-channel modes fail these tasks.
- Audit trajectories: Outcome-only scores inflate success.
- Match model to runtime: Cross-pairings collapse scores.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces” (arXiv:2606.09426v3, submitted June 8, 2026). All numbers are taken from the paper’s text, tables and figures; i10X did not rerun the experiments. The paper is distributed under arXiv’s non-exclusive license, so i10X does not reproduce its figures; the chart was drawn by i10X from numbers reported in interface ablation and best pairing.
i10x Perspective
WeaveBench’s sharpest result is the ablation: strip away either GUI or CLI and PassRate collapses to a few percent. The hybrid agent is not a convenience feature. It is the capability.
