Remote Labor Index: Claude Fable 5.2 Completes 41% of Real Freelance Jobs. Here’s What the Number Leaves Out

Claude Fable 5.2 leads Scale Labs' Remote Labor Index at 41.25%, meaning reviewers accepted its work on 99 of 240 real paid freelance projects, up from a 2.5% top score a year ago. The jump is real, but briefs were complete and client calls and revisions were excluded.

Christopher Ort
By Christopher Ort · 7 min read
Share
A home office desk with a laptop, printed project briefs and sketches, and an empty chair pushed back in morning light.
Image: AI-generated, i10X
On this page
  1. Quick Take
  2. In plain English
  3. What happened
  4. Background: what the Remote Labor Index is
  5. How the test works (and where the 41% comes from)
  6. The fine print
  7. Pros and cons
  8. What X is saying
  9. Grok's take
  10. Critical analysis
  11. What's still open
  12. About the analysis
  13. i10x Perspective

A year ago, the best AI agent could finish 2.5% of real, paid freelance jobs to a client’s standard. Today the top score on the Remote Labor Index (RLI) is 41.25%, held by Anthropic’s Claude Fable 5.2. A chart of that climb went viral on X this weekend, with predictions that “white collar work will get a wake-up in 2027.” Before anyone updates their CV, it is worth looking at what the benchmark measures, and what it leaves out.

i10X Quick Take

Quick Take

  1. The number: Claude Fable 5.2 leads Scale Labs’ RLI leaderboard at 41.25%, ahead of Opus 5.5 (21.25%) and GPT 6 Astra (20.83%).
  2. What it means: In 99 of 240 real freelance projects, human reviewers judged the AI’s deliverable at least as good as the work a paying client actually accepted from a professional.
  3. The trend: The best score at launch in October 2025 was 2.5% (Manus). The RLI site and the Center for AI Safety (CAIS) say the automation rate has been doubling roughly every three months.
  4. The catch: Projects come with complete briefs and input files. Client calls, revision rounds, teamwork and live systems are excluded by design.
  5. Why it matters: RLI is the closest thing to a “real jobs” benchmark, so its numbers shape the debate about AI and remote work.

In plain English

Researchers bought real freelance jobs, like design, video or data work, that people had already been paid for online. They then gave the same jobs to AI systems and had experts judge whether a client would accept the AI’s result. The best AI now passes on about 4 in 10 of those jobs, up from fewer than 3 in 100 a year ago. But the AI got a perfect job description and every file it needed, and it never had to talk to a client or redo work after feedback. So the test shows real progress, not that 41% of freelancers are about to be replaced.

What happened

Anthropic released Claude Fable 5.2 and Mythos 5.2 on November 9 (see our launch coverage). The same day, Fable 5.2 took first place on the RLI leaderboard run by Scale Labs with CAIS. Scale announced on X that the model “successfully automates 41.3% of projects, nearly doubling the performance of the next-best model Opus 5.5.”

The story trended again over the weekend after AI commentator @kimmonismus posted that “in 13 months, the best AI agent went from completing 2.5% of real freelance projects on the Remote Labor Index to 41.3% with Claude Fable 5.2.” The post drew more than 1,300 likes and a long reply thread arguing about what the number really covers.

Background: what the Remote Labor Index is

RLI was introduced by Scale AI and CAIS in an arXiv paper on October 30, 2025. According to the-decoder and Sulat, the dataset includes 240 completed projects from 358 verified Upwork professionals across 23 of Upwork’s 64 work domains, worth about $144,000 in total. Examples include game development, product design, architecture, data analysis and video animation.

Scale’s leaderboard defines the main metric, Automation Rate, as “the percentage of projects where an AI’s deliverable is judged at least as good as the human standard,” meaning whether “a ‘reasonable client’ would accept the work.” The leaderboard also reports an Elo score against a fixed human baseline of 1,000 and the dollar value of projects completed.

How the test works (and where the 41% comes from)

  • Same inputs: The AI gets the original brief and all input files, then works in a computer environment and returns a deliverable.
  • Human grading: Trained evaluators compare it with the professional’s real deliverable. A majority of three decides pass or fail, according to an analysis by Pebblous.
  • The math: Anthropic’s system card lists the result as 99 of 240, which is 41.25%; Scale’s site and Anthropic’s announcement round it to 41.3%. Anthropic lists it as a High effort score.
  • Job size: Per the RLI paper as summarized by Pebblous, the median project took a human 11.5 hours and cost $200; the means are 28.9 hours and $632.6.

Current top of the Scale leaderboard (automation rate):

RankModelAutomation rate
1Claude Fable 5.241.25%
2Claude Opus 5.521.25%
3GPT 6 Astra20.83%
4Claude Fable 5.117.92%
5Gemini Argon 417.08%
6Claude Fable 515.83%
7Claude Opus 4.88.33%
8Codex GPT 5.56.25%

The fine print

Pebblous, a data company, published a detailed critique worth reading in full. Three points stand out:

  • Which denominator? The paper says quantitative evaluation uses a private set of 230 projects, but every leaderboard score divides evenly by 240, not 230. Pebblous found no published explanation.
  • The scaffold changed: Citing Epoch AI, it notes the evaluation environment (Linux with tools like GIMP and FreeCAD, Veo 3 for video tasks) was significantly upgraded partway through the time series, so the jump from 2.5% is not a pure model-to-model comparison.
  • Pre-cut jobs: 550 candidate projects were narrowed to 240. Sourcing rules required that work be doable by one freelancer, need no client interaction, need no live client systems and be gradable on the spot.

Scale itself says RLI “does not represent the entire digital economy.” Sulat adds that CAIS’s suggestion that AIs may soon do all remote work faster and cheaper is “a projection from the recent pace, not a finding from the 41.3% result itself.”

Pros and cons

For users

  • Pro: For well-defined tasks with clear inputs, such as a logo variant, a data cleanup or a short animation, AI agents are now often good enough to deliver acceptable work.
  • Con: The 59% of failures are on projects that were already cleanly specified. Messier real requests will likely fail more often.

For developers

  • Pro: RLI is an end-to-end benchmark with real economic stakes, a useful counterweight to quiz-style tests.
  • Con: Scores depend heavily on the scaffold, tools and time limits. Comparing your agent to the leaderboard requires matching those conditions.

For companies

  • Pro: Firms that already write precise briefs and acceptance criteria can start routing suitable freelance-type tasks to agents.
  • Con: A 41% automation rate is not a 41% cost cut. Scoping, revisions and quality control stay human work, and the benchmark does not measure them.

What X is saying

The viral version came from @kimmonismus:

Forecaster Peter Wildeford put the jump in context on launch day:

The replies zeroed in on what is missing. “‘Briefs provided and client calls excluded’ is doing a lot of work there. Most of those 29 hours were never the typing. They were working out what the client actually meant,” wrote @Halfcycleai. “The 59% left is the client calls, the revisions, and the invoice that never gets paid,” quipped @achronon. And @ChakibBouabd argued: “The jump is real; a 2027 white-collar timeline needs those messy parts measured.”

Grok’s take

Explaining the benchmark on X, Grok wrote: “Remote Labor Index (RLI) is a CAIS + Scale AI benchmark: AI agents tackle 240 real Upwork freelance projects (design, code, 3D, analysis, etc.). Automation rate = % of projects human evaluators accept as client-ready quality vs the original pro deliverable.” In another reply, it cautioned that a widely shared “700%” projection was “the uncapped exponential extrapolation” of the trend, while “the index maxes at 100%.”

Critical analysis

The trend is real and large. Going from single digits to 41% on paid, expert-graded projects in about a year is not a rounding error, and the RLI design is more honest than most benchmarks because the bar is “would a client accept this?”

But three things limit what the number can tell you. The test population is filtered toward self-contained, file-in, file-out work. The environment around the model has changed over time, which inflates long-run comparisons. And the hardest parts of freelancing, figuring out what the client wants and iterating on feedback, are deliberately excluded. Pebblous puts it neatly: 41.25% applies to “the jobs that made it through those nine sieves, rather than 41.25% of remote labor.”

The extrapolations deserve the most skepticism. A doubling every three months is an average of very uneven steps, Pebblous calculates, from about 35 days to about 231 days per doubling depending on the period. Projecting that curve to 100% by a specific month is a guess, not a measurement.

What’s still open

  • Whether Scale and CAIS clarify the 230 versus 240 denominator question.
  • How OpenAI’s and Google’s next models score under the same scaffold.
  • Whether a future RLI version adds client interaction or revision rounds.
  • How much of the gain comes from the model versus better tools in the test environment.

About the analysis

Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated image. Read our AI Use Policy. Spotted an error? Request a correction.

This independent i10x analysis is based on Scale Labs’ public RLI leaderboard, the original RLI paper, coverage by the-decoder and Sulat, a critical analysis by Pebblous (which cites Epoch AI and Anthropic’s system card), and public posts on X. Leaderboard values as published at the time of writing.

i10x Perspective

The Remote Labor Index is the right kind of benchmark asked the wrong kind of question on X. It shows that AI agents can now finish a large share of well-specified, solo, deliverable-based jobs. It does not show that 41% of freelance work is gone. For freelancers, the practical takeaway is to move up the stack: the scoping, client handling and judgment the benchmark leaves out are exactly the parts that stay valuable. For companies, the lesson is less flashy but more useful: write better briefs, because that is what makes agents work.

Christopher Ort

Christopher Ort

Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). He is the author of “The Style Penalty”, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.

Timeline

What happened before

  1. Release

    Claude Haiku 5.5

    1M context · $0.1 / $0.5 per 1M tokens

    support.claude.com (source, opens in a new tab)
  2. Release

    Claude Sonnet 5.5

    1M context · $2 / $10 per 1M tokens

    support.claude.com (source, opens in a new tab)
  3. Release

    Claude Opus 5.5

    1M context · $4 / $20 per 1M tokens

    First model of the Claude 5.5 family; performs at the level of Claude Fable 5.1 on most work and costs 40% less to run than Opus 5.

    anthropic.com (source, opens in a new tab)
  4. Release

    Claude Fable 5.1 & Mythos 5.1

    1M context · $10 / $50 per 1M tokens

    Anthropic's most advanced models for coding and knowledge work. Price/context shown for Claude Fable 5.1 (models overview).

    support.claude.com (source, opens in a new tab)
  5. Release

    Claude Opus 5

    1M context · $5 / $25 per 1M tokens

    support.claude.com (source, opens in a new tab)
Full Claude timeline ›

Editor's picks

  1. RedAmon flaws let signed-in users run root commands in the AI pentest agent container
  2. CVE-2026-129877: DeepAudit exposes LLM API keys and Git tokens to unauthenticated attackers
  3. CVE-2026-129875: REBUILD 4.5.0–4.5.4 password hash disclosure via MCP QueryRecords
  4. South Korea Weighs Nuclear Power to Meet AI Electricity Demand
  5. UK and US Partner to Accelerate Nuclear Fusion with AI