Benchmarks & EvalsAnthropicClaude
Evidence check
  • Preprint
  • No code released
  • Not independently replicated

Stealth Model Gestalt Beats Human Designers on Contra’s Frontier Design Benchmark

A stealth model called Gestalt leads Contra Labs' Frontier Design Benchmark with 1,931 Elo and a 90% win rate, ahead of human designers and Claude Fable 5.2. What the 4,311 blind votes show, which new models are missing, and the AI design patterns designers spot.

Christopher Ort
By i10X News Desk · Reviewed by Christopher Ort · 4 min read
Share
A hand with a pencil hovers between two printed website layouts on a design studio table, choosing one.
Image: AI-generated, i10X
On this page
  1. Quick Take
  2. How the benchmark works
  3. The board moved within hours of launch
  4. Two tiers, by our count
  5. Taste beats the checklist, and what builders can use
  6. What is still open

Contra Labs, the research arm of the freelance platform Contra, released its Frontier Design Benchmark on 18 November: 33 website and app briefs, mostly adapted from paid client jobs, judged in blind pairs by professional designers. First place on the live leaderboard goes to a model nobody can use yet. “Gestalt”, a stealth entry whose developer Contra does not name, sits at 1,931 Elo with a 90% win rate, ahead of the human reference designs at 1,739 and 74%. According to Contra Labs, it is the first model rated above those human designs, which is why design teams should care even before it ships.

i10X Quick Take

Quick Take

  1. The test: 27 briefs adapted from Contra jobs plus 6 real projects from startup partners, 35 designers, 4,311 blind votes.
  2. The ranking: Gestalt first, human designers second, then Claude Fable 5.2 (1,713) and Claude Opus 5.5 (1,691). Google’s best entry, Gemini 4 Carbon, is 12th.
  3. Our calculation: a 192-point Elo lead over humans implies Gestalt should win about three of four head-to-head picks. Contra’s own count, 20 of 28 against the human designs, fits that.
  4. Missing: GPT-6.2 Astra and Gemini 4 Neon, both released this week, are not on the board yet.

How the benchmark works

Contra rewrote 27 real website jobs from its marketplace for fictional clients and added six projects that startups had actually hired Contra designers for, with their own brand files. Models ran as agents in sandboxes through BenchFlow’s SDK, with OpenRouter as provider and, where possible, each lab’s own agent harness. For the human baseline, Contra paid designers to go through moodboards and feedback rounds before building their “golden” version.

Designers then compared two anonymous outputs for the same brief and picked the better one. Contra converts the votes into Elo ratings with a Bradley-Terry model. When it split the panel into two random halves, the two rankings agreed at a Spearman correlation of 0.95, a sign the ordering is not just noise.

The board moved within hours of launch

When Contra Labs announced the benchmark, humans were still on top:

About twenty minutes later, a Contra team member posted the Gestalt preview:

The research post lists Gestalt at 1,928 Elo with 378 wins in 423 votes. The live board now shows 1,931 and 90% on 420 votes, so the figures are still being updated. Current standings for selected entries:

RankEntryMakerEloWin rate
1Gestalt (stealth)Not disclosed1,93190%
2Human designersContra designers1,73974%
3Claude Fable 5.2Anthropic1,71373%
4Claude Opus 5.5Anthropic1,69173%
5GPT-6.1 SolOpenAI1,66870%
6Claude Sonnet 5.5Anthropic1,66769%
7Kimi K3.1 ProMoonshot AI1,66371%
8GPT-6 AstraOpenAI1,63866%
9Qwen3.8 MaxAlibaba1,43439%
12Gemini 4 CarbonGoogle1,36636%
15Grok 4.7xAI1,24723%
17DeepSeek V4 ProDeepSeek1,12112%

Two tiers, by our count

Below Gestalt and the humans, the board splits cleanly. Six models sit between 1,638 and 1,713 Elo: three from Anthropic, two from OpenAI and Moonshot’s open-weights Kimi K3.1 Pro. The next entry, Qwen3.8 Max, is 204 points lower at 1,434. That makes Kimi K3.1 Pro the only open-weights model in the top group, and puts both Gemini models, Grok 4.7 and DeepSeek V4 Pro in the lower one.

Two caveats apply. The newest releases have not been tested: GPT-6.2 Astra launched on 17 November and Gemini 4 Neon on 19 November, so the weak Google showing may not last. And Contra is not a neutral referee: it says it already works with frontier labs to improve their models’ design output, and it offers evaluation data to labs as a service. Gestalt itself has no named developer, no API and no price, so nobody outside Contra can check the result.

Taste beats the checklist, and what builders can use

The most useful finding for teams shipping AI-built sites is a disagreement. Contra also grades every page on a rubric of concrete design rules. There, the human designs score 73.6 and rank only sixth of 16, behind the Claude models (79.3 to 83.8) and both GPT entries. Designers still preferred the human pages in 74% of votes. The rubric catches errors like low-contrast buttons, while the voters reward overall judgment.

Contra’s “Slop Detector” also lists patterns that make a page look machine-made. Worth checking before you publish a generated landing page:

  • A small capitalized label above every heading that repeats it. Contra found this on at least half the sections of 93% of AI pages in its training set, against 11% of human pages.
  • One italic word in the main headline, and section numbers like 01, 02, 03.
  • Every block in its own card, a habit of Gemini 4 Argon (97% of its pages) and Gemini 4 Carbon (94%).
  • Lab fingerprints: Claude models favor a horizontal arrow on links, GPT models a diagonal one.

What is still open

  • Who built Gestalt, and whether and when it becomes available.
  • Whether GPT-6.2 Astra, Gemini 4 Neon and other new models get added, and how the gap to humans changes.
  • Independent replication of the votes by anyone outside Contra and BenchFlow.
Christopher Ort

Christopher Ort

Christopher Ort is the editor of i10X News at i10x.ai. He has worked hands-on with AI tools, models and agent workflows for years and leads i10X’s coverage of AI launches, funding and research. He is the author of “The Style Penalty”, an SSRN study on how frontier AI models judge AI-written CVs, covered by heise online and t3n.

Timeline

What happened before

  1. Release

    Claude Haiku 5.5

    1M context · $0.1 / $0.5 per 1M tokens

    support.claude.com (source, opens in a new tab)
  2. Release

    Claude Sonnet 5.5

    1M context · $2 / $10 per 1M tokens

    support.claude.com (source, opens in a new tab)
  3. Release

    Claude Opus 5.5

    1M context · $4 / $20 per 1M tokens

    First model of the Claude 5.5 family; performs at the level of Claude Fable 5.1 on most work and costs 40% less to run than Opus 5.

    anthropic.com (source, opens in a new tab)
  4. Release

    Claude Fable 5.1 & Mythos 5.1

    1M context · $10 / $50 per 1M tokens

    Anthropic's most advanced models for coding and knowledge work. Price/context shown for Claude Fable 5.1 (models overview).

    support.claude.com (source, opens in a new tab)
  5. Release

    Claude Opus 5

    1M context · $5 / $25 per 1M tokens

    support.claude.com (source, opens in a new tab)
Full Claude timeline ›

Editor's picks

  1. Georgia Tech’s Academic AI Playbook: 20 Lessons From 67 Real Courses, Free to Reuse
  2. Arcfront’s $1B Run Rate: What the AI Data Startup Confirms and What It Doesn’t
  3. GPT-6.2 Astra Tops Blueprint-Bench 2, 70% Above May Leader GPT-5.5
  4. Deutsche Telekom Puts 100 Million Euros Into Mistral for Sovereign AI in Germany
  5. Inkling 2 Mini: Thinking Machines’ Looped Open Model Tops Qwen on Coding Tests