{"id":7137,"date":"2026-11-11T08:02:58","date_gmt":"2026-11-11T08:02:58","guid":{"rendered":"https:\/\/i10xnews.kinsta.cloud\/?p=7137"},"modified":"2026-11-11T08:02:58","modified_gmt":"2026-11-11T08:02:58","slug":"coding-agents-expert-tasks-expertcrowd-swe-study","status":"publish","type":"post","link":"https:\/\/i10x.ai\/news\/coding-agents-expert-tasks-expertcrowd-swe-study","title":{"rendered":"Study: Claude Opus 5.5 and GPT-6 Astra Tie on Expert Coding Tasks"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">\u26a1 Quick Take<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>The finding:<\/strong> On 100 coding tasks written by practising engineers for real open-source codebases, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) lead a field of seven frontier models, and the gap between them is not statistically significant.<\/li><li><strong>Why it stands out:<\/strong> Unlike crowded leaderboards, the benchmark spreads models over 35 points in four distinct tiers. No task was fully solved by every model, and full-credit runs are rare: 26.6% for the leader, 1.0% for Qwen 3.8 Max and Grok 4.6.<\/li><li><strong>What it means:<\/strong> The two leaders are complementary rather than ranked: Opus 5.5 solves more tasks at least once, GPT-6 Astra is more consistent and needs about a fifth of the steps. Pick by workflow, not by one number.<\/li><\/ul>\n\n\n<p class=\"wp-block-paragraph\">Coding agents now get hired to change real codebases, yet the benchmarks used to compare them are fraying. OpenAI stopped reporting SWE-bench Verified in February 2026 after an audit found problems in 59.4% of 138 checked tasks, and on the authored DeepSWE v1.1 leaderboard four frontier models sit within 0.9 points of each other. A new technical report from Toptal ExpertCrowd, &#8220;ExpertCrowd-SWE: Measuring Frontier Coding Agents on Expert-Authored Engineering Tasks in Unfamiliar Codebases&#8221; by Josip Vrdoljak and six co-authors, tries a different recipe: tickets written by engineers who know the codebase, graded the way a maintainer reviews a pull request.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd2c What the study shows<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Clear tiers:<\/strong> Claude Opus 5.5 scores 67.3% and GPT-6 Astra 63.7%, followed by Claude Fable 5.1 (46.5%) and Gemini 3.8 Flash (40.0%). Qwen 3.8 Max, DeepSeek V4.1 Flash and Grok 4.6 form a statistically indistinguishable group at 32.2\u201333.5%. The seven-model mean is 45.1%.<\/li><li><strong>The top two are tied:<\/strong> Opus 5.5 leads GPT-6 Astra by 3.6 points, but the paired 95% interval runs from \u22121.4 to +9.7 (p = 0.20). Each wins many tasks: 53 to 41, with 6 ties. Every other model trails Opus 5.5 significantly (p &lt; 0.001).<\/li><li><strong>Lots of headroom:<\/strong> On 80 of the 100 tasks no model earns full credit on all five runs. The hardest task, a SystemVerilog export for the CirC compiler, left all seven models between 2.3% and 2.7%.<\/li><li><strong>Breadth versus reliability:<\/strong> Opus 5.5 reaches full credit at least once on 47 tasks versus 36 for GPT-6 Astra, but GPT-6 Astra hits full credit on all five runs for 15 tasks versus 9, and its scores vary less within a task (SD 7.3 vs. 13.6 points).<\/li><li><strong>Effort is not capability:<\/strong> GPT-6 Astra needs a median of 33 agent steps per run and reaches 95% of Opus 5.5&#8217;s score with about a fifth of its steps (Opus 5.5: 160). Gemini 3.8 Flash takes the most steps (166) and ranks fourth.<\/li><li><strong>Maintainer rules bite:<\/strong> 7.8% of all runs were capped by a failed must-pass criterion and 2.3% zeroed by a failed gate such as &#8220;leave the existing tests untouched&#8221;. The strongest models lost the most, but the ranking did not change.<\/li><\/ul>\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1344\" height=\"768\" src=\"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/11\/chart-ec-swe-scores.webp\" alt=\"Bar chart: mean task score on ExpertCrowd-SWE: Claude Opus 5.5 67.3%, GPT-6 Astra 63.7%, Claude Fable 5.1 46.5%, Gemini 3.8 Flash 40.0%, Qwen 3.8 Max 33.5%, DeepSeek V4.1 Flash 32.5%, Grok 4.6 32.2%.\" class=\"wp-image-7136\" srcset=\"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-ec-swe-scores.webp 1344w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-ec-swe-scores-300x171.webp 300w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-ec-swe-scores-1024x585.webp 1024w, https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/chart-ec-swe-scores-768x439.webp 768w\" sizes=\"auto, (max-width: 1344px) 100vw, 1344px\" \/><figcaption class=\"wp-element-caption\">i10X chart based on data from Vrdoljak et al. (2026), Toptal ExpertCrowd, arXiv:2611.13246, Table 3.<\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Model<\/th><th>Score (95% CI)<\/th><th>Full-credit runs<\/th><th>Median steps per run<\/th><th>Median container time<\/th><\/tr><\/thead><tbody><tr><td>Claude Opus 5.5<\/td><td>67.3 \u00b1 4.7<\/td><td>26.6%<\/td><td>160<\/td><td>80 min<\/td><\/tr><tr><td>GPT-6 Astra<\/td><td>63.7 \u00b1 5.5<\/td><td>23.4%<\/td><td>33<\/td><td>36 min<\/td><\/tr><tr><td>Claude Fable 5.1<\/td><td>46.5 \u00b1 3.7<\/td><td>8.6%<\/td><td>127<\/td><td>56 min<\/td><\/tr><tr><td>Gemini 3.8 Flash<\/td><td>40.0 \u00b1 4.3<\/td><td>3.8%<\/td><td>166<\/td><td>40 min<\/td><\/tr><tr><td>Qwen 3.8 Max<\/td><td>33.5 \u00b1 3.3<\/td><td>1.0%<\/td><td>137<\/td><td>100 min<\/td><\/tr><tr><td>DeepSeek V4.1 Flash<\/td><td>32.5 \u00b1 3.7<\/td><td>3.0%<\/td><td>160<\/td><td>51 min<\/td><\/tr><tr><td>Grok 4.6<\/td><td>32.2 \u00b1 3.5<\/td><td>1.0%<\/td><td>73<\/td><td>69 min<\/td><\/tr><\/tbody><\/table><figcaption class=\"wp-element-caption\">Selected columns from Vrdoljak et al. (2026), arXiv:2611.13246, Tables 3 and 6. Gemini 3.8 Flash scored over 99 tasks.<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">\u2705 Why it matters: grading like a maintainer<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The design targets the two known weak spots of mined benchmarks. Contamination: tasks are new changes that were never published as fixes, 76 of 100 are held out, and every task was compared with 499,647 records from 93 public benchmark datasets without finding a duplicate. Grading: instead of inheriting the tests of one historical patch, each task has a weighted rubric written by its author, with mostly executable checks through public interfaces. Each task image only builds if the reference solution scores exactly 1.0 and both the untouched repo and a comment-only change score 0. Nearly half the tasks also ship alternative, lazy and &#8220;cheat&#8221; patches the grader must classify correctly, and LLM judges carry just 0.85% of the total weight. The statistics are careful too: paired permutation tests with Holm correction, and both task-sampling and run-to-run intervals. The former are two to ten times wider, which the authors rightly point out most leaderboards understate.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83e\uddea Method in brief<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Tasks:<\/strong> 100 multi-file change requests across 87 open-source projects and 11 languages (Go leads with 36). Median ticket: 148 words; median reference solution: 471 changed lines across 4.5 files. 82 base commits date from August or September 2026.<\/li><li><strong>Authors:<\/strong> 39 engineers from Toptal&#8217;s network contributed; the final tasks come from 17 of them. Each task was quality-checked by a senior engineer.<\/li><li><strong>Models and harness:<\/strong> Seven models, each run through the HUD platform with its provider&#8217;s native tool interface at default high reasoning effort, with no system prompt and no vendor product such as Claude Code or Codex CLI.<\/li><li><strong>Runs:<\/strong> Five runs per model and task with a 1,000-step limit, 3,495 scored runs collected from October 1 to 10, 2026. Runs that hit step or token limits were rerun (27 for Opus 5.5, 55 for Gemini 3.8 Flash, 2 for Grok 4.6).<\/li><\/ul>\n\n\n<h2 class=\"wp-block-heading\">\u2753 What remains open<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Harness differences:<\/strong> Each provider gets its own tool interface, so part of a gap may reflect tool design rather than the model. A shared single-tool rerun is listed as future work.<\/li><li><strong>One reasoning setting:<\/strong> Each model ran at its platform&#8217;s default high effort. Effort was not swept, so other settings could change both scores and step counts.<\/li><li><strong>Sample size:<\/strong> With 100 tasks, the authors say differences under about 5 points between neighbouring models should not be over-interpreted.<\/li><li><strong>Public code underneath:<\/strong> The repositories are public and likely in training data, so EC-SWE tests problem solving in familiar-looking code.<\/li><li><strong>Who measures:<\/strong> This is a vendor technical report, not peer reviewed. The authors work for Toptal ExpertCrowd, which runs the leaderboard and supplies expert talent; no funding or competing-interest statement is given. Only 24 tasks are public, so outsiders can verify part of the measurement.<\/li><\/ul>\n\n\n<h2 class=\"wp-block-heading\">\ud83e\udded Context: after the SWE-bench era<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">SWE-bench mined 2,294 GitHub fixes and became the standard; later variants added languages, multimodal tasks and private splits. Research showed models reproduce SWE-bench gold patches far more often than comparable unseen code, and OpenAI&#8217;s own audits in 2026 pushed the field toward authored tasks. DeepSWE (113 original tasks, one bash tool for all models) is the closest neighbour. Both benchmarks agree on the top two but diverge below, most clearly for DeepSeek V4.1 Flash: third on DeepSWE, sixth here. The authors call that comparison indicative, not controlled. We covered similar harness effects in <a href=\"\/news\/coding-agent-harness-vs-model-benchmarks\">harness versus model in coding benchmarks<\/a> and real-world agent requests in <a href=\"\/news\/boba-bench-coding-agents-request-chains-study\">BOBA-Bench<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udee0\ufe0f What it means for engineering teams<\/h2>\n\n\n<ul class=\"wp-block-list\"><li><strong>Match the model to the workflow:<\/strong> If you can run several attempts and verify them, Opus 5.5&#8217;s breadth helps. If one attempt has to land, GPT-6 Astra&#8217;s consistency and lower step count matter more.<\/li><li><strong>Budget for effort:<\/strong> Step counts varied five-fold and container time from 36 to 100 minutes, with little link to quality. Track cost and latency next to accuracy.<\/li><li><strong>Guard your tests:<\/strong> Gates caught runs that edited tests or broke stated constraints. Make &#8220;existing tests untouched&#8221; a hard check in your own agent pipelines.<\/li><li><strong>Expect partial work:<\/strong> Most strong runs earned at least 40% of the credit but few were merge-ready. Plan human review for multi-file changes.<\/li><\/ul>\n\n\n<p class=\"wp-block-paragraph\">More on the <a href=\"\/news\/category\/studies\/agent-research\">Agents research<\/a> page and the <a href=\"\/news\/product\/claude\">Claude<\/a> hub.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u270d\ufe0f About the analysis<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This article is based on the full text of Vrdoljak, Singh, Steunebrink, Yarotsky, Lorenzo, Nicoletti and Pigott (2026), &#8220;ExpertCrowd-SWE: Measuring Frontier Coding Agents on Expert-Authored Engineering Tasks in Unfamiliar Codebases,&#8221; arXiv:2611.13246v1 (cs.AI, technical report, submitted November 9, 2026), including the appendix. All numbers come from the paper. The chart was redrawn by i10X from Table 3; the paper is distributed under the arXiv non-exclusive license, so no original figures were reused. No funding or competing-interest statement is included; the authors are affiliated with Toptal ExpertCrowd. AI tools assisted with research and drafting; selection, fact-checking and final editing were done by the i10X editorial team.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\ud83d\udd2d i10x Perspective<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The headline is less &#8220;Claude beats GPT&#8221; than &#8220;benchmarks can still separate frontier agents if tasks look like real tickets and grading looks like real review.&#8221; The caveat is obvious: a talent marketplace grading AI coders has its own interests, and 76 hidden tasks ask for trust. Still, the design choices, from calibrated graders to honest uncertainty intervals, are ones every coding leaderboard should copy.<\/p>\n\n","protected":false},"excerpt":{"rendered":"<p>A new benchmark of 100 engineer-written tickets splits seven frontier coding agents into four tiers. Claude Opus 5.5 and GPT-6 Astra lead, statistically tied.<\/p>\n","protected":false},"author":2,"featured_media":7135,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"paper_title":"ExpertCrowd-SWE: Measuring Frontier Coding Agents on Expert-Authored Engineering Tasks in Unfamiliar Codebases","paper_url":"https:\/\/arxiv.org\/abs\/2611.13246","pdf_url":"https:\/\/arxiv.org\/pdf\/2611.13246","doi":"","arxiv_id":"2611.13246","authors":"Josip Vrdoljak, Akash Singh, Bas Steunebrink, Alex Yarotsky, John Lorenzo, Achille Nicoletti, Sam Pigott","institutions":"Toptal ExpertCrowd","paper_date":"2026-11-09","venue":"arXiv (cs.AI), technical report, version 1","peer_reviewed":false,"code_url":"","data_url":"https:\/\/huggingface.co\/datasets\/ExpertCrowd\/ec-swe","funding_coi":"No funding statement. Authors are affiliated with Toptal ExpertCrowd, which runs the benchmark leaderboard; evaluation platform by HUD. No competing-interests statement.","figure_license":"Own chart redrawn from Table 3 (no original figures reused)","paper_license":"arXiv non-exclusive license","replication":"partial","study_models":"Claude Opus 5.5, GPT-6 Astra, Claude Fable 5.1, Gemini 3.8 Flash, Qwen 3.8 Max, DeepSeek V4.1 Flash, Grok 4.6 (HUD harness, provider-native tools)","study_sample":"100 expert-authored multi-file tasks (87 open-source projects, 11 languages; 24 public, 76 held out); 5 runs per model and task, 3,495 scored runs","footnotes":"","_yoast_wpseo_primary_category":"","_nx_sec_severity":"","_nx_sec_affected":"","_nx_sec_status":"","_nx_sec_kicker":""},"categories":[53,52],"tags":[],"companies":[39,46,37,38,40],"products":[51,49,48],"class_list":["post-7137","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-benchmarks-evals","category-studies","company-anthropic","company-deepseek","company-google","company-open-ai","company-x-ai","product-claude","product-gemini","product-grok"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Claude Opus 5.5 vs GPT-6 Astra: Expert Coding Benchmark - i10X News<\/title>\n<meta name=\"description\" content=\"ExpertCrowd-SWE study: on 100 engineer-written coding tasks, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) tie at the top; five models trail far behind.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Claude Opus 5.5 vs GPT-6 Astra: Expert Coding Benchmark - i10X News\" \/>\n<meta property=\"og:description\" content=\"ExpertCrowd-SWE study: on 100 engineer-written coding tasks, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) tie at the top; five models trail far behind.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study\" \/>\n<meta property=\"og:site_name\" content=\"i10X News\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-11T08:02:58+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/11\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"1344\" \/>\n\t<meta property=\"og:image:height\" content=\"768\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"Christopher Ort\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@i10x_ai\" \/>\n<meta name=\"twitter:site\" content=\"@i10x_ai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Christopher Ort\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"NewsArticle\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study\"},\"author\":{\"name\":\"Christopher Ort\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/person\\\/d410bb7ee72febefca2c9c369939219a\"},\"headline\":\"Study: Claude Opus 5.5 and GPT-6 Astra Tie on Expert Coding Tasks\",\"datePublished\":\"2026-11-11T08:02:58+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study\"},\"wordCount\":1325,\"publisher\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp\",\"articleSection\":[\"Benchmarks &amp; Evals\",\"Studies\"],\"inLanguage\":\"en-US\",\"dateModified\":\"2026-11-11T08:02:58+00:00\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study\",\"url\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study\",\"name\":\"Claude Opus 5.5 vs GPT-6 Astra: Expert Coding Benchmark - i10X News\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp\",\"datePublished\":\"2026-11-11T08:02:58+00:00\",\"description\":\"ExpertCrowd-SWE study: on 100 engineer-written coding tasks, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) tie at the top; five models trail far behind.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp\",\"contentUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/11\\\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp\",\"width\":1344,\"height\":768,\"caption\":\"Illustration: i10X\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/coding-agents-expert-tasks-expertcrowd-swe-study#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/i10x.ai\\\/news\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Study: Claude Opus 5.5 and GPT-6 Astra Tie on Expert Coding Tasks\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#website\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\",\"name\":\"i10X News\",\"description\":\"Stay updated with the latest news and announcements from i10X.\",\"publisher\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/i10x.ai\\\/news?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\",\"name\":\"i10X\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/i10x-logo-180.png\",\"contentUrl\":\"https:\\\/\\\/i10xnews.kinsta.cloud\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/i10x-logo-180.png\",\"width\":180,\"height\":180,\"caption\":\"i10X\"},\"image\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/i10x_ai\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/i10x-ai\",\"https:\\\/\\\/instagram.com\\\/i10x.ai\",\"https:\\\/\\\/www.youtube.com\\\/@i10X_ai_official\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#\\\/schema\\\/person\\\/d410bb7ee72febefca2c9c369939219a\",\"name\":\"Christopher Ort\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"contentUrl\":\"https:\\\/\\\/i10x.ai\\\/news\\\/wp-content\\\/uploads\\\/2026\\\/10\\\/christopher-ort-192.webp\",\"caption\":\"Christopher Ort\"},\"description\":\"Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU). He is the author of \u201cThe Style Penalty\u201d, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.\",\"url\":\"https:\\\/\\\/i10x.ai\\\/news\\\/author\\\/christopher-ort\",\"sameAs\":[\"https:\\\/\\\/i10x.ai\\\/blog\\\/author\\\/christopher-ort\"],\"worksFor\":{\"@id\":\"https:\\\/\\\/i10x.ai\\\/news\\\/#organization\"},\"alumniOf\":{\"@type\":\"EducationalOrganization\",\"name\":\"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU)\",\"url\":\"https:\\\/\\\/www.fau.eu\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Claude Opus 5.5 vs GPT-6 Astra: Expert Coding Benchmark - i10X News","description":"ExpertCrowd-SWE study: on 100 engineer-written coding tasks, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) tie at the top; five models trail far behind.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study","og_locale":"en_US","og_type":"article","og_title":"Claude Opus 5.5 vs GPT-6 Astra: Expert Coding Benchmark - i10X News","og_description":"ExpertCrowd-SWE study: on 100 engineer-written coding tasks, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) tie at the top; five models trail far behind.","og_url":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study","og_site_name":"i10X News","article_published_time":"2026-11-11T08:02:58+00:00","og_image":[{"width":1344,"height":768,"url":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/11\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp","type":"image\/webp"}],"author":"Christopher Ort","twitter_card":"summary_large_image","twitter_creator":"@i10x_ai","twitter_site":"@i10x_ai","twitter_misc":{"Written by":"Christopher Ort","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"NewsArticle","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#article","isPartOf":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study"},"author":{"name":"Christopher Ort","@id":"https:\/\/i10x.ai\/news\/#\/schema\/person\/d410bb7ee72febefca2c9c369939219a"},"headline":"Study: Claude Opus 5.5 and GPT-6 Astra Tie on Expert Coding Tasks","datePublished":"2026-11-11T08:02:58+00:00","mainEntityOfPage":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study"},"wordCount":1325,"publisher":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"image":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage"},"thumbnailUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp","articleSection":["Benchmarks &amp; Evals","Studies"],"inLanguage":"en-US","dateModified":"2026-11-11T08:02:58+00:00"},{"@type":"WebPage","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study","url":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study","name":"Claude Opus 5.5 vs GPT-6 Astra: Expert Coding Benchmark - i10X News","isPartOf":{"@id":"https:\/\/i10x.ai\/news\/#website"},"primaryImageOfPage":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage"},"image":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage"},"thumbnailUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp","datePublished":"2026-11-11T08:02:58+00:00","description":"ExpertCrowd-SWE study: on 100 engineer-written coding tasks, Claude Opus 5.5 (67.3%) and GPT-6 Astra (63.7%) tie at the top; five models trail far behind.","breadcrumb":{"@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#primaryimage","url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp","contentUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/11\/coding-agents-expert-tasks-expertcrowd-swe-study-title.webp","width":1344,"height":768,"caption":"Illustration: i10X"},{"@type":"BreadcrumbList","@id":"https:\/\/i10xnews.kinsta.cloud\/coding-agents-expert-tasks-expertcrowd-swe-study#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/i10x.ai\/news"},{"@type":"ListItem","position":2,"name":"Study: Claude Opus 5.5 and GPT-6 Astra Tie on Expert Coding Tasks"}]},{"@type":"WebSite","@id":"https:\/\/i10x.ai\/news\/#website","url":"https:\/\/i10x.ai\/news","name":"i10X News","description":"Stay updated with the latest news and announcements from i10X.","publisher":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/i10x.ai\/news?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/i10x.ai\/news\/#organization","name":"i10X","url":"https:\/\/i10x.ai\/news\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10x.ai\/news\/#\/schema\/logo\/image\/","url":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/i10x-logo-180.png","contentUrl":"https:\/\/i10xnews.kinsta.cloud\/wp-content\/uploads\/2026\/10\/i10x-logo-180.png","width":180,"height":180,"caption":"i10X"},"image":{"@id":"https:\/\/i10x.ai\/news\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/i10x_ai","https:\/\/www.linkedin.com\/company\/i10x-ai","https:\/\/instagram.com\/i10x.ai","https:\/\/www.youtube.com\/@i10X_ai_official"]},{"@type":"Person","@id":"https:\/\/i10x.ai\/news\/#\/schema\/person\/d410bb7ee72febefca2c9c369939219a","name":"Christopher Ort","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","url":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","contentUrl":"https:\/\/i10x.ai\/news\/wp-content\/uploads\/2026\/10\/christopher-ort-192.webp","caption":"Christopher Ort"},"description":"Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU). He is the author of \u201cThe Style Penalty\u201d, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.","url":"https:\/\/i10x.ai\/news\/author\/christopher-ort","sameAs":["https:\/\/i10x.ai\/blog\/author\/christopher-ort"],"worksFor":{"@id":"https:\/\/i10x.ai\/news\/#organization"},"alumniOf":{"@type":"EducationalOrganization","name":"Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU)","url":"https:\/\/www.fau.eu"}}]}},"_links":{"self":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/7137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/comments?post=7137"}],"version-history":[{"count":1,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/7137\/revisions"}],"predecessor-version":[{"id":7144,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/posts\/7137\/revisions\/7144"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/media\/7135"}],"wp:attachment":[{"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/media?parent=7137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/categories?post=7137"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/tags?post=7137"},{"taxonomy":"company","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/companies?post=7137"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/i10x.ai\/news\/wp-json\/wp\/v2\/products?post=7137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}