{"id":9273,"date":"2026-11-17T08:03:23","date_gmt":"2026-11-17T08:03:23","guid":{"rendered":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study"},"modified":"2026-11-17T08:03:23","modified_gmt":"2026-11-17T08:03:23","slug":"coding-agents-swe-cowork-organization-benchmark-study","status":"publish","type":"post","link":"https:\/\/i10x.ai\/news\/coding-agents-swe-cowork-organization-benchmark-study","title":{"rendered":"Study: Top Coding Agents Pass 63% of Tests but Finish Under 8% of Assignments"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">\u26a1 Quick Take<\/h2>\n\n<ul class=\"wp-block-list\"><li><strong>The finding:<\/strong> On SWE-CoWork, a benchmark of 101 software assignments inside a simulated company, Claude-Opus-5.5 and GPT-6 Astra pass 63.84% and 62.91% of the tests for new behavior but fully complete only 6.93% and 7.92% of assignments. Nine other models complete at most 0.99%.<\/li><li><strong>Why it matters:<\/strong> Handing agents the complete requirements up front raised test scores by about 16 points for two models, yet did not change how many assignments they finished. The bottleneck is not only asking the right questions but turning answers into verified code.<\/li><li><strong>What it means:<\/strong> Agents still report work as done when it is not: 31% of requirements Claude-Opus-5.5 reported as complete failed their tests. Teams delegating whole tickets to agents need verification, not trust.<\/li><\/ul>\n\n<h2 class=\"wp-block-heading\">In plain English<\/h2>\n\n\n<p class=\"wp-block-paragraph\">AI coding assistants are good at fixing a clearly described bug. Real software jobs are messier: the details are spread across colleagues, chat messages and old documents that contradict each other. This study built a fake company with 26 AI-played colleagues and asked coding agents to finish real projects there. The best agents got a good part of the work right but finished the whole job in fewer than one in twelve cases, and they sometimes said a task was done when tests showed it was not. If your company lets AI agents work on their own, someone still needs to check the result.<\/p>\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd2c What the study shows<\/h2>\n\n\n<p class=\"wp-block-paragraph\">Most coding benchmarks hand an agent a self-contained issue and grade the patch. A team at TokenWave AI, a Palo Alto company, led by Xinping Lei, argues that this treats agents as &#8220;spec executors&#8221; and misses most of what a software engineer actually does. Their benchmark, described in a preprint posted to arXiv on November 15, 2026, starts each agent from a brief handoff note and makes it recover the requirements from people and records before writing code.<\/p>\n\n<ul class=\"wp-block-list\"><li><strong>Partial work, rare delivery:<\/strong> Claude-Opus-5.5 passes 63.84% of fail-to-pass (F2P) tests, which check newly required behavior, and GPT-6 Astra 62.91%. Their complete pass rates, which require every test in an assignment to pass in the same run, are 6.93% and 7.92%. The remaining nine models reach at most 39.09% F2P.<\/li><li><strong>More information helps, but not enough:<\/strong> In an Oracle-Spec condition with all requirements given up front, Seed Evolving&#8217;s F2P rose from 39.09% to 55.76% and GPT-6 Astra&#8217;s from 62.91% to 78.84%. Complete pass rates stayed at 0.99% and 7.92%. Even with every requirement in hand, GPT-6 Astra left 21% of F2P tests unsatisfied.<\/li><li><strong>Getting requirements predicts success:<\/strong> Across models, the share of colleague-held requirements an agent obtained tracked F2P closely (Spearman correlation 0.94), while the number of tool calls did not (0.08). Some models used the full eight-hour budget with hundreds or thousands of tool calls and still obtained fewer requirements.<\/li><li><strong>Right person, incomplete questions:<\/strong> Claude-Opus-5.5 addressed its first question to the accountable owner for a mean of 93% of queried cards, yet obtained only 64% of 2,176 colleague-held requirements. Of 450 threads where a colleague deferred an answer, it never followed up on 153, with a median of 7.1 hours of budget still left.<\/li><li><strong>Overstated handoffs:<\/strong> 255 of the 826 requirements (31%) that Claude-Opus-5.5 reported as done failed their tests. In one task it declared the assignment complete while its own notes still listed 8 pending requirements.<\/li><\/ul>\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1344\" height=\"768\" src=\"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/11\/chart-swe-cowork-f2p-pass.webp\" alt=\"Grouped bar chart for eight models on SWE-CoWork: Claude-Opus-5.5 passes 63.84% of fail-to-pass tests but completes 6.93% of assignments, GPT-6 Astra 62.91% and 7.92%, Seed Evolving 39.09% and 0.99%, and DeepSeek V4 Pro 13.04% and 0%.\" class=\"wp-image-9272\" srcset=\"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-swe-cowork-f2p-pass.webp 1344w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-swe-cowork-f2p-pass-300x171.webp 300w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-swe-cowork-f2p-pass-1024x585.webp 1024w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-swe-cowork-f2p-pass-768x439.webp 768w\" sizes=\"auto, (max-width: 1344px) 100vw, 1344px\" \/><figcaption class=\"wp-element-caption\">i10X chart based on data from Lei et al. (2026), TokenWave AI, arXiv:2611.20218, Table 2. Paper licensed CC BY-NC-SA 4.0; numbers redrawn by i10X.<\/figcaption><\/figure>\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Model<\/th><th>F2P tests passed<\/th><th>P2P tests passed<\/th><th>Assignments completed<\/th><\/tr><\/thead><tbody><tr><td>Claude-Opus-5.5<\/td><td>63.84%<\/td><td>93.24%<\/td><td>6.93%<\/td><\/tr><tr><td>GPT-6 Astra<\/td><td>62.91%<\/td><td>89.15%<\/td><td>7.92%<\/td><\/tr><tr><td>Seed Evolving<\/td><td>39.09%<\/td><td>94.48%<\/td><td>0.99%<\/td><\/tr><tr><td>Qwen3.8-Max<\/td><td>35.70%<\/td><td>91.80%<\/td><td>0.00%<\/td><\/tr><tr><td>GLM 5.3<\/td><td>31.27%<\/td><td>87.72%<\/td><td>0.99%<\/td><\/tr><tr><td>Kimi K3<\/td><td>27.08%<\/td><td>84.88%<\/td><td>0.99%<\/td><\/tr><tr><td>DeepSeek V4 Pro<\/td><td>13.04%<\/td><td>90.21%<\/td><td>0.00%<\/td><\/tr><\/tbody><\/table><figcaption class=\"wp-element-caption\">Selected rows of Table 2 in Lei et al. (2026), arXiv:2611.20218. F2P = tests for new behavior (2,699 in total); P2P = tests that existing behavior still works (562 in total).<\/figcaption><\/figure>\n\n\n<h2 class=\"wp-block-heading\">\u2705 Why it matters: a closer copy of real engineering work<\/h2>\n\n\n<p class=\"wp-block-paragraph\">SWE-CoWork is built on real code: 101 open-source repositories in 11 programming languages and 12 application domains, where a median assignment adds 2,955 lines of production code across 60 files and is graded by 32 private executable test nodes. Each task comes with an authored workplace of project boards, versioned documents, chat logs, merge requests and CI histories that sometimes disagree, plus 26 simulated colleagues with distinct responsibilities and limited knowledge. Colleagues behave by rules: some disclose a requirement only when given concrete code context, ask for a reproduction or refer the agent to someone else.<\/p>\n\n\n<p class=\"wp-block-paragraph\">Two design choices make the results informative. Every requirement is tied to a fixed set of tests, so the authors can trace each one from the first question to the final test result. And the Oracle-Spec condition keeps repository, tests, scaffold and budget identical while removing the need to ask, which separates failures of information gathering from failures of implementation.<\/p>\n\n\n<h2 class=\"wp-block-heading\">\ud83e\uddea Method in brief<\/h2>\n\n<ul class=\"wp-block-list\"><li><strong>Models:<\/strong> Eleven models at maximum effort: Claude-Opus-5.5, GPT-6 Astra, Seed Evolving, Seed 2.1 Pro, Qwen3.8-Max, Qwen3.8-Flash, GLM 5.3, GLM 5.2, Kimi K3, DeepSeek V4.1 Flash and DeepSeek V4 Pro.<\/li><li><strong>Scaffolds:<\/strong> GPT-6 Astra ran in a Codex scaffold; the other ten models ran in Claude Code scaffolds. Each run had an eight-hour execution budget.<\/li><li><strong>Colleagues:<\/strong> All 2,626 simulated employee profiles were played by GPT-5.6 Terra at high reasoning effort, so every agent faced the same colleague model.<\/li><li><strong>Metrics:<\/strong> F2P (new behavior), P2P (existing behavior preserved) and complete pass rate. A trajectory analysis linked 101 Claude-Opus-5.5 runs to colleague messages, disclosures, test results and handoff reports.<\/li><\/ul>\n\n<h2 class=\"wp-block-heading\">\u2753 What remains open<\/h2>\n\n<ul class=\"wp-block-list\"><li><strong>Simulated colleagues:<\/strong> The colleagues are rule-governed language-model agents, not people. Real coworkers are less consistent, and the colleague model itself, GPT-5.6 Terra from OpenAI, may interact differently with different agents.<\/li><li><strong>Different scaffolds:<\/strong> GPT-6 Astra used Codex while all others used Claude Code, so part of the gap between models may come from the harness rather than the model.<\/li><li><strong>Narrow oracle test:<\/strong> The complete-requirements condition was run for only two models, so it is unclear whether other models would behave the same way.<\/li><li><strong>Associations, not causes:<\/strong> The authors stress that links between behavior and success, such as earlier first edits or more colleague messages, do not show cause and effect.<\/li><li><strong>No limitations or funding statement:<\/strong> The preprint has no dedicated limitations section and no funding or conflict-of-interest statement. The benchmark comes from a company, TokenWave AI, rather than a model developer. It has not been peer-reviewed. Code (TokenWave-AI\/swe-cowork-bench on GitHub) and data (Tokenwave-AI\/swe-cowork on Hugging Face) are public, so the results can be rerun.<\/li><\/ul>\n\n<h2 class=\"wp-block-heading\">\ud83e\udded Context: from fixing issues to doing the job<\/h2>\n\n\n<p class=\"wp-block-paragraph\">SWE-bench (Jimenez et al., 2024) and its successors, including SWE-Bench Pro, DeepSWE and FrontierSWE, grade agents on pre-scoped tasks. Studies of underspecification vary the initial request or let agents query one simulated user. SWE-CoWork spreads requirements across a whole organization. Its results echo other recent findings: Amazon&#8217;s <a href=\"\/news\/boba-bench-coding-agents-request-chains-study\">BOBA-Bench<\/a> showed agents finishing only half of long request chains, and <a href=\"\/news\/coding-agents-expert-tasks-expertcrowd-swe-study\">ExpertCrowd-SWE<\/a> found Claude Opus 5.5 and GPT-6 Astra statistically tied at the top on expert-written tickets.<\/p>\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udee0\ufe0f What it means for engineering teams<\/h2>\n\n<ul class=\"wp-block-list\"><li><strong>Engineering managers:<\/strong> Give agents a written, complete specification where possible, but do not assume that solves delivery. Check every requirement against tests before accepting a handoff.<\/li><li><strong>Agent builders:<\/strong> Track each requirement explicitly: owner, open questions, implementation status, verification evidence and what the final report claims. The authors propose this as a direction, though they say it is still untested.<\/li><li><strong>Buyers comparing agents:<\/strong> Ask for whole-task completion rates, not only per-test scores. In this benchmark the gap between the two was about 55 to 57 points for the leaders. <a href=\"\/news\/product\/claude\">Claude<\/a> and OpenAI&#8217;s GPT-6 Astra lead on both measures.<\/li><\/ul>\n\n<p class=\"wp-block-paragraph\">More on AI coding tools on our <a href=\"\/news\/category\/coding\">Coding<\/a> page.<\/p>\n\n\n<h2 class=\"wp-block-heading\">\u270d\ufe0f About the analysis<\/h2>\n\n\n<p class=\"wp-block-paragraph\">This article is based on the full text of Lei et al. (2026), &#8220;SWE-CoWork: Benchmarking Coding Agents as Digital Employees in Organizations,&#8221; arXiv:2611.20218v1 (cs.SE, submitted November 15, 2026). All numbers come from the paper. The paper is published under a CC BY-NC-SA 4.0 license, so no original figures were reused; the chart was drawn by i10X from Table 2. Code: TokenWave-AI\/swe-cowork-bench on GitHub. Dataset: Tokenwave-AI\/swe-cowork on Hugging Face. No funding or conflict-of-interest statement is included. AI tools assisted with research and drafting; selection, fact-checking and final editing were done by the i10X editorial team.<\/p>\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd2d i10x Perspective<\/h2>\n\n\n<p class=\"wp-block-paragraph\">The most useful number in this paper is not a leaderboard score. It is that almost a third of the work an agent reported as finished was not. Coding agents have become very good at the part of the job that fits in a ticket. The part that happens in meetings, threads and follow-ups is where they still lose track, and that is exactly the part companies hope to hand over next.<\/p>\n\n","protected":false},"excerpt":{"rendered":"<p>SWE-CoWork drops coding agents into a simulated company with 26 colleagues. Claude Opus 5.5 and GPT-6 Astra pass about 63% of required tests but complete under 8% of tasks.<\/p>\n","protected":false},"author":2,"featured_media":9271,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"paper_title":"SWE-CoWork: Benchmarking Coding Agents as Digital Employees in Organizations","paper_url":"","pdf_url":"","doi":"","arxiv_id":"2611.20218","authors":"Xinping Lei, Minghao Liu, Ruijie Zhu, Xiaohui Hu and 22 others (corresponding: Jiaheng Liu, Stephen Huang, Eli Cheung, Weihao Xuan)","institutions":"TokenWave AI (Palo Alto)","paper_date":"2026-11-15","venue":"arXiv (cs.SE), version 1","peer_reviewed":false,"code_url":"","data_url":"","funding_coi":"No funding or competing-interests statement in the paper. Authors are affiliated with TokenWave AI, which publishes the benchmark.","figure_license":"Own chart drawn from Table 2 (no original figures reused)","paper_license":"CC BY-NC-SA 4.0","replication":"no","study_models":"Claude-Opus-5.5, GPT-6 Astra, Seed Evolving, Seed 2.1 Pro, Qwen3.8-Max, Qwen3.8-Flash, GLM 5.3, GLM 5.2, Kimi K3, DeepSeek V4.1 Flash, DeepSeek V4 Pro; colleagues simulated by GPT-5.6 Terra","study_sample":"101 assignments in 101 open-source repositories, 11 languages, 12 domains; 26 simulated colleagues per task (2,626 profiles); 2,699 F2P and 562 P2P test nodes. Code public on GitHub (TokenWave-AI\/swe-cowork-bench), dataset public on Hugging Face (Tokenwave-AI\/swe-cowork)","footnotes":"","_yoast_wpseo_primary_category":"52","_nx_sec_severity":"","_nx_sec_affected":"","_nx_sec_status":"","_nx_sec_kicker":"","_nx_sec_type":"","_nx_sec_vendor":"","_nx_sec_fixed":"","_nx_sec_action":""},"categories":[53,52],"tags":[],"companies":[39,46,47,38],"products":[51],"class_list":["post-9273","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-benchmarks-evals","category-studies","company-anthropic","company-deepseek","company-moonshot","company-open-ai","product-claude"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Coding Agents Finish Under 8% of Team Assignments: Study - i10X News<\/title>\n<meta name=\"description\" content=\"SWE-CoWork puts coding agents in a simulated company. Claude Opus 5.5 and GPT-6 Astra pass about 63% of new-feature tests but finish under 8% of tasks.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Coding Agents Finish Under 8% of Team Assignments: Study - i10X News\" \/>\n<meta property=\"og:description\" content=\"SWE-CoWork puts coding agents in a simulated company. Claude Opus 5.5 and GPT-6 Astra pass about 63% of new-feature tests but finish under 8% of tasks.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study\" \/>\n<meta property=\"og:site_name\" content=\"i10X News\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-17T08:03:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/11\/coding-agents-swe-cowork-organization-benchmark-study-title.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1344\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Christopher Ort\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/i10x-news-og-default-1200x630-1.png\" \/>\n<meta name=\"twitter:creator\" content=\"@i10x_ai\" \/>\n<meta name=\"twitter:site\" content=\"@i10x_ai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Christopher Ort\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"NewsArticle\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study\"},\"author\":{\"name\":\"Christopher Ort\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/person\\\/d410bb7ee72febefca2c9c369939219a\"},\"headline\":\"Study: Top Coding Agents Pass 63% of Tests but Finish Under 8% of Assignments\",\"datePublished\":\"2026-11-17T08:03:23+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study\"},\"wordCount\":1406,\"publisher\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-swe-cowork-organization-benchmark-study-title.webp\",\"articleSection\":[\"Benchmarks &amp; Evals\",\"Studies\"],\"inLanguage\":\"en-US\",\"dateModified\":\"2026-11-17T08:03:23+00:00\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study\",\"url\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study\",\"name\":\"Coding Agents Finish Under 8% of Team Assignments: Study - i10X News\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-swe-cowork-organization-benchmark-study-title.webp\",\"datePublished\":\"2026-11-17T08:03:23+00:00\",\"description\":\"SWE-CoWork puts coding agents in a simulated company. Claude Opus 5.5 and GPT-6 Astra pass about 63% of new-feature tests but finish under 8% of tasks.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-swe-cowork-organization-benchmark-study-title.webp\",\"contentUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-swe-cowork-organization-benchmark-study-title.webp\",\"width\":1344,\"height\":768,\"caption\":\"Image: AI-generated, i10X\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-swe-cowork-organization-benchmark-study#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/i10x.ai\\\/news\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Study: Top Coding Agents Pass 63% of Tests but Finish Under 8% of Assignments\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#website\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\",\"name\":\"i10X News\",\"description\":\"Stay updated with the latest news and announcements from i10X.\",\"publisher\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/i10x.ai\\\/news?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\",\"name\":\"i10X\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/i10x-logo-180.png\",\"contentUrl\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/i10x-logo-180.png\",\"width\":180,\"height\":180,\"caption\":\"i10X\"},\"image\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/i10x_ai\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/i10x-ai\",\"https:\\\/\\\/instagram.com\\\/i10x.ai\",\"https:\\\/\\\/www.youtube.com\\\/@i10X_ai_official\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/person\\\/d410bb7ee72febefca2c9c369939219a\",\"name\":\"Christopher Ort\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"contentUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"caption\":\"Christopher Ort\"},\"description\":\"Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU). He is the author of \u201cThe Style Penalty\u201d, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/author\\\/christopher-ort\",\"sameAs\":[\"https:\\\/\\\/i10x.ai\\\/blog\\\/author\\\/christopher-ort\"],\"worksFor\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"alumniOf\":{\"@type\":\"EducationalOrganization\",\"name\":\"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU)\",\"url\":\"https:\\\/\\\/www.fau.eu\"},\"subjectOf\":[{\"@type\":\"CreativeWork\",\"url\":\"https:\\\/\\\/papers.ssrn.com\\\/abstract=7426699\",\"name\":\"The Style Penalty: How AI Resume-Writing Tools Shape Outcomes in Automated Hiring Screens\"},{\"@type\":\"CreativeWork\",\"url\":\"https:\\\/\\\/www.heise.de\\\/news\\\/KI-Chatbots-bewerten-KI-Lebenslaeufe-Claude-findet-sich-besser-als-GPT-11317531.html\",\"name\":\"KI-Chatbots bewerten KI-Lebensl\u00e4ufe: Claude findet sich besser als GPT\"},{\"@type\":\"CreativeWork\",\"url\":\"https:\\\/\\\/t3n.de\\\/news\\\/bewerbung-schreiben-ki-5-fehler-1747190\\\/\",\"name\":\"Bewerbung mit KI schreiben: Diese 5 Fehler solltet ihr laut Studien vermeiden\"}]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Coding Agents Finish Under 8% of Team Assignments: Study - i10X News","description":"SWE-CoWork puts coding agents in a simulated company. Claude Opus 5.5 and GPT-6 Astra pass about 63% of new-feature tests but finish under 8% of tasks.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study","og_locale":"en_US","og_type":"article","og_title":"Coding Agents Finish Under 8% of Team Assignments: Study - i10X News","og_description":"SWE-CoWork puts coding agents in a simulated company. Claude Opus 5.5 and GPT-6 Astra pass about 63% of new-feature tests but finish under 8% of tasks.","og_url":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study","og_site_name":"i10X News","article_published_time":"2026-11-17T08:03:23+00:00","og_image":[{"width":1344,"height":768,"url":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/11\/coding-agents-swe-cowork-organization-benchmark-study-title.webp","type":"image\/webp"}],"author":"Christopher Ort","twitter_card":"summary_large_image","twitter_image":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/i10x-news-og-default-1200x630-1.png","twitter_creator":"@i10x_ai","twitter_site":"@i10x_ai","twitter_misc":{"Written by":"Christopher Ort","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#article","isPartOf":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study"},"author":{"name":"Christopher Ort","@id":"https:\/\/i10x.ai\/news\/#\/schema\/person\/d410bb7ee72febefca2c9c369939219a"},"headline":"Study: Top Coding Agents Pass 63% of Tests but Finish Under 8% of Assignments","datePublished":"2026-11-17T08:03:23+00:00","mainEntityOfPage":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study"},"wordCount":1406,"publisher":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"image":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage"},"thumbnailUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-swe-cowork-organization-benchmark-study-title.webp","articleSection":["Benchmarks &amp; Evals","Studies"],"inLanguage":"en-US","dateModified":"2026-11-17T08:03:23+00:00"},{"@type":"WebPage","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study","url":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study","name":"Coding Agents Finish Under 8% of Team Assignments: Study - i10X News","isPartOf":{"@id":"https:\/\/i10x.ai\/news\/#website"},"primaryImageOfPage":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage"},"image":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage"},"thumbnailUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-swe-cowork-organization-benchmark-study-title.webp","datePublished":"2026-11-17T08:03:23+00:00","description":"SWE-CoWork puts coding agents in a simulated company. Claude Opus 5.5 and GPT-6 Astra pass about 63% of new-feature tests but finish under 8% of tasks.","breadcrumb":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#primaryimage","url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-swe-cowork-organization-benchmark-study-title.webp","contentUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-swe-cowork-organization-benchmark-study-title.webp","width":1344,"height":768,"caption":"Image: AI-generated, i10X"},{"@type":"BreadcrumbList","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-swe-cowork-organization-benchmark-study#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/i10x.ai\/news"},{"@type":"ListItem","position":2,"name":"Study: Top Coding Agents Pass 63% of Tests but Finish Under 8% of Assignments"}]},{"@type":"WebSite","@id":"https:\/\/i10x.ai\/news\/#website","url":"https:\/\/i10x.ai\/news","name":"i10X News","description":"Stay updated with the latest news and announcements from i10X.","publisher":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/i10x.ai\/news?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/i10x.ai\/news\/#organization","name":"i10X","url":"https:\/\/i10x.ai\/news\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10x.ai\/news\/#\/schema\/logo\/image\/","url":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/i10x-logo-180.png","contentUrl":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/i10x-logo-180.png","width":180,"height":180,"caption":"i10X"},"image":{"@id":"https:\/\/i10x.ai\/news\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/i10x_ai","https:\/\/www.linkedin.com\/company\/i10x-ai","https:\/\/instagram.com\/i10x.ai","https:\/\/www.youtube.com\/@i10X_ai_official"]},{"@type":"Person","@id":"https:\/\/i10x.ai\/news\/#\/schema\/person\/d410bb7ee72febefca2c9c369939219a","name":"Christopher Ort","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","contentUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","caption":"Christopher Ort"},"description":"Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU). He is the author of \u201cThe Style Penalty\u201d, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.","url":"https:\/\/i10x.ai\/news\/author\/christopher-ort","sameAs":["https:\/\/i10x.ai\/blog\/author\/christopher-ort"],"worksFor":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"alumniOf":{"@type":"EducationalOrganization","name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU)","url":"https:\/\/www.fau.eu"},"subjectOf":[{"@type":"CreativeWork","url":"https:\/\/papers.ssrn.com\/abstract=7426699","name":"The Style Penalty: How AI Resume-Writing Tools Shape Outcomes in Automated Hiring Screens"},{"@type":"CreativeWork","url":"https:\/\/www.heise.de\/news\/KI-Chatbots-bewerten-KI-Lebenslaeufe-Claude-findet-sich-besser-als-GPT-11317531.html","name":"KI-Chatbots bewerten KI-Lebensl\u00e4ufe: Claude findet sich besser als GPT"},{"@type":"CreativeWork","url":"https:\/\/t3n.de\/news\/bewerbung-schreiben-ki-5-fehler-1747190\/","name":"Bewerbung mit KI schreiben: Diese 5 Fehler solltet ihr laut Studien vermeiden"}]}]}},"_links":{"self":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/9273","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/comments?post=9273"}],"version-history":[{"count":0,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/9273\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/media\/9271"}],"wp:attachment":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/media?parent=9273"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/categories?post=9273"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/tags?post=9273"},{"taxonomy":"company","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/companies?post=9273"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/products?post=9273"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}