Quick Take
- The finding: Even the strongest models, Gemini-3.1-pro and Kimi-k2.6, reach only about 30.67% average pass rate on Workflow-GYM’s professional GUI workflows.
- Contrast with OS-World: The same class of SOTA models exceeds 70% on OS-World, which uses more general software and shorter tasks.
- What it means: Domain-specific tools and 30–110-step workflows are a different problem than clicking through consumer apps.
GUI agent benchmarks still lean on general-purpose software and short tasks. ByteDance Seed, M-A-P and collaborators introduce Workflow-GYM to test whether agents can operate specialized professional software end-to-end for economically valuable work.
What the study shows
Tasks span ease levels with step ranges of 30–44 (38.2%), 45–60 (47.0%) and 61–110 (14.8%). Domain experts manually verify each environment. Multiple SOTA models under different agent frameworks were tested.
- Pass@3 still low: Best pass@3 is 41.42%; most models stay under 30%.
- Failure modes: Agents omit workflow stages, propagate errors, drift from objectives and misunderstand professional software.
- Looping hotspots: Geography & Environment tasks are 2.48× enriched among looping failures despite being 10.95% of tasks; Scientific Computing is 1.38×; Data Analysis is underrepresented (0.53×).
Why it matters: enterprise software is the product
Enterprises care about CAD, GIS, scientific and analytics tools, not only browsers and office suites. Workflow-GYM shows that gap clearly.
Method in brief
Multi-stage validation including expert manual execution. Project: workflow-gym.github.io.
What remains open
- Environment specificity: Results depend on the chosen professional apps.
- Framework effects: Agent scaffolding interacts with model skill.
Context
Complements OS-World-style GUI tests. See Benchmarks & Evals.
What it means for GUI-agent vendors
- Test domain software: Consumer GUI scores will not transfer.
- Detect loops early: Geography and scientific apps especially.
- Guard stage completeness: Stage omission is a primary failure mode.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields” (arXiv:2606.11042v4, submitted June 9, 2026). All numbers are taken from the paper’s text, tables and figures; i10X did not rerun the experiments. The chart was redrawn by i10X from Table 2 and OS-World comparison of the paper, which is published under a CC BY-NC-ND 4.0 license.
i10x Perspective
Thirty percent success on the workflows people actually get paid to run is a sober checkpoint. GUI agents that dazzle on demo apps still get lost in professional tools. Closing that gap is where the next wave of agent value lives.
