Society & EconomyAnthropicChatGPT
Evidence check
  • Preprint
  • No code released
  • Not independently replicated

Study: Newer Chatbots Give More Legal Advice, Refer Fewer to Lawyers

A UC Berkeley audit of 23 chatbots built from 65 court cases finds 58.4% of answers gave legal advice reserved for lawyers, with newer models crossing the line more often.

Christopher Ort
By Christopher Ort · 7 min read
Share
A man sits at a table at night holding his phone, with a pile of legal paperwork spread out in front of him.
Image: AI-generated, i10X
On this page
  1. Quick Take
  2. In plain English
  3. What the study shows
  4. Why it matters: a test grounded in case law
  5. Method in brief
  6. What remains open
  7. Context: from passing the bar to practicing law
  8. What it means for users, AI companies and the legal profession
  9. About the analysis
  10. i10x Perspective
i10X Quick Take

Quick Take

  1. The finding: In an audit of 23 chatbot models released between 2023 and 2026, 58.4% of 26,910 responses gave legal advice that US courts reserve for licensed lawyers, and newer models did so significantly more often.
  2. Why it matters: The test is built from 65 real court decisions on the unauthorized practice of law in immigration, bankruptcy, personal injury, foreclosure and family law. Outside attorneys who reviewed sampled transcripts found advice that was incomplete, misleading and at times dangerous.
  3. What it means: Chatbots are shifting from “ask a lawyer” to acting like one. Referrals to a lawyer dropped by a median 12.6 percentage points between the first and second turn of a conversation.

In plain English

In the United States, only licensed lawyers may apply the law to your personal situation, a rule meant to protect people from bad advice. Researchers at UC Berkeley turned 65 real court cases, in which people were hurt by advice from someone who was not a lawyer, into test conversations for 23 popular chatbots. More than half of the chatbot answers crossed that line, and newer chatbots crossed it more often than older ones. Lawyers who read some of the conversations found advice that left out important facts and could cause real harm, for example in immigration and foreclosure cases. A chatbot can help you understand a legal topic, but for your own case you should still talk to a lawyer or a legal aid group.

What the study shows

People with legal problems are turning to chatbots: the paper notes that self-represented filings in US federal civil courts have risen more than 50% above prior averages since ChatGPT’s release. At the same time, AI companies have tuned their models to refuse less. The authors point to Anthropic, whose 2023 constitution told Claude to avoid the impression of giving specific legal advice, while the 2026 version says Claude can be like a brilliant friend with the knowledge of a doctor, lawyer and financial advisor. Nel Escher, Deirdre Mulligan and colleagues at UC Berkeley and Swarthmore College asked whether this shift means chatbots now do work that the law reserves for lawyers.

  • Most answers cross the line: The authors classify 58.4% of all responses as unauthorized practice of law (UPL). Rates range from 20.3% for Meta’s Llama 3.3 70B to 88.5% for DeepSeek R1. OpenAI’s chatbots have the highest provider average at 76.1%.
  • Newer models give more advice: A trend test across model release dates finds a statistically significant rise in UPL (z = +2.58, permutation p = 0.0038, one-tailed). The paper’s fitted trend is +11.0 percentage points per year.
  • A policy change shows up in the data: Claude Fable 5 (June 2026) gave more legal advice than Claude Haiku 4.5 (October 2025) in 60 of 65 cases, by 35.6 percentage points on average. Anthropic removed legal advice as a hazard from its constitution in January 2026. The authors caution that the two models also differ in other ways, such as size.
  • Saying where you live invites more advice: UPL rates were higher for 21 of 23 models when the user named their state, by a median 4.4 points.
  • No domain is safe: Immigration had the lowest UPL rate of the five domains, yet 52% of those responses still contained legal advice.
  • Fewer referrals as the chat goes on: 81.0% of responses pointed users to a lawyer or legal aid, but only 61.5% with a strong recommendation. Referral rates ranged from 96.0% for Google’s Gemma 2-2B to 38.8% for OpenAI’s chat-latest. From the first to the second turn, referrals fell for 22 of 23 models. A downward trend over release dates (z = -1.43, p = 0.077) was not statistically significant.
Horizontal bar chart of the share of chatbot responses labeled as unauthorized practice of law in UPLBench: highest model DeepSeek R1 88.5%, OpenAI chatbots mean 76.1%, all 23 models 58.4%, lowest model Llama 3.3 70B 20.3%.
i10X chart based on data from Escher et al. (2026), arXiv:2611.17622, Section 5.1. CC BY 4.0.
MeasureResult
Responses classified as legal advice (all 23 models)58.4%
Lowest and highest model20.3% (Llama 3.3 70B), 88.5% (DeepSeek R1)
OpenAI chatbots, mean UPL rate76.1%
Claude Fable 5 vs. Claude Haiku 4.5+35.6 points, higher in 60 of 65 cases
Responses referring users to a lawyer or legal aid81.0% (strong referrals: 61.5%)
Change in referrals, turn 1 to turn 2Median -12.6 points, lower for 22 of 23 models
Selected results from Section 5 of Escher et al. (2026), arXiv:2611.17622. 26,910 classified responses.

Why it matters: a test grounded in case law

Most legal AI evaluations measure capability, such as bar exam scores. This one measures compliance with an existing legal line. Each of the 65 benchmark items comes from a court decision that ruled on unauthorized practice, and each has a rubric that tracks the court’s own reasoning, so a response counts as legal advice when it does what a judge called UPL, such as suggesting language for a pleading. Conversations run over three turns, because people rarely ask just once, and each case is tested with and without the user’s location, three times per model.

The authors also guarded against shortcuts. A pseudonymization pass replaced names, addresses, case numbers and amounts so chatbots could not simply recall the original case. The automatic classifier, the open-weight Qwen3.8-27B from a provider outside the audited set, was validated against two human annotators (macro F1 of 0.84; the humans agreed with each other at Cohen’s kappa 0.78). Trend tests shuffle release dates at the model level, so answers from the same model are not treated as independent.

Method in brief

  • Models: 17 open-weight chat models from Meta, OpenAI, Google and DeepSeek with over one million Hugging Face downloads, released from October 2023 to June 2026, plus six closed models: the default free and top paid models of Google, OpenAI and Anthropic as of July 2026, queried through their APIs.
  • Benchmark: 65 UPL cases from 33 state and federal courts in five high-stakes domains, each turned into a first-person scenario plus three follow-up requests that solicit conduct the court called UPL.
  • Scale: 23 models x 65 cases x 3 turns x 2 location conditions x 3 repetitions = 26,910 responses, each labeled for legal advice and for referral to outside help.
  • Expert review: Four practicing attorneys (immigration, foreclosure and two personal injury) each reviewed 10 transcripts, 120 responses in total, for accuracy of the law and quality of the legal service.

What remains open

  • API, not apps: Closed models were tested through their APIs, which can behave differently from the consumer apps people actually use.
  • History is hard to rebuild: Old closed models cannot be reconstructed faithfully, so the long-term trend rests mostly on open-weight checkpoints.
  • Machine labels: Most labels come from an automatic classifier whose agreement with each annotator (kappa 0.62 and 0.59) is below the humans’ agreement with each other.
  • Courts may see AI differently: The rubrics apply reasoning courts used for human nonlawyers. Whether courts would treat chatbot output the same way is open, and an insurer’s lawsuit against OpenAI turns on exactly that question.
  • Scripted prompts and a small expert panel: Real users phrase questions differently, and the expert review covers 40 transcripts. The study detects whether advice was given; it does not grade how good each answer was across all responses.
  • Status: Preprint submitted to an ACM journal, not yet peer reviewed. The paper does not link a public release of the benchmark or code and includes no funding statement.

Context: from passing the bar to practicing law

Earlier work asked whether GPT-4 could pass the bar exam, and LegalBench measured legal reasoning on 162 tasks. Safety sets such as Do-Not-Answer lump legal advice in with other risky requests without separating legal information from legal advice, and Trident-Bench imagines a lawyer asking for help with something unethical. UPLBench instead asks whether chatbots, answering ordinary people, cross the line that courts have drawn. It fits a pattern we saw in a recent audit of chatbot mental health advice, where who asks shapes what the model says.

  • Users: Treat chatbot answers on a legal problem as background reading. Be careful with sensitive details: attorneys flagged chatbots asking about deportation history and visa status without any of the confidentiality protections a lawyer provides.
  • AI companies: Helpfulness settings are a legal and product decision, not just a tuning choice. Anthropic, Google and OpenAI, the maker of ChatGPT, all had models that crossed the line in this audit, and OpenAI’s chat-latest referred users to lawyers least often.
  • Bar authorities and regulators: Because every failure traces back to a specific court case, state bars can isolate results for their own jurisdiction instead of waiting for new rules.

More on our Society & Economy studies page.

About the analysis

Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated image. Read our AI Use Policy. Spotted an error? Request a correction.

This article is based on the full text of Escher, Elliott, Qian, Sharifullina, Guhr, Davis and Mulligan (2026), “Auditing Chatbots for Incursion into Professional Legal Practice,” arXiv:2611.17622v1 (cs.CY, submitted November 13, 2026). All numbers come from the paper. The chart was redrawn by i10X from figures stated in Section 5.1 of the paper, which is published under a CC BY 4.0 license. The authors state they used Claude Opus 5.5 for editing and Claude Code for scaling their pipeline, and note that they reviewed Claude’s output critically, “particularly its frequent protests concerning Anthropic’s framing in this paper.” No conflicts of interest stated. AI tools assisted with research and drafting; selection, fact-checking and final editing were done by the i10X editorial team.

i10x Perspective

The debate about AI and access to justice usually runs on hope and anecdotes. This audit adds hard numbers: the industry has quietly moved from refusing legal questions to answering them, and the answers often skip the questions a real lawyer would ask first. Whether the fix is stricter models or new rules for AI legal help, that decision should be made in the open, not inside a model’s settings.

Christopher Ort

Christopher Ort

Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). He is the author of “The Style Penalty”, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.

Timeline

What happened before

  1. Release

    GPT-6.1 Terra

    1.05M context · $1 / $5 per 1M tokens

    openai.com (source, opens in a new tab)
  2. Release

    GPT-6.1 Sol

    1.05M context · $2 / $10 per 1M tokens

    developers.openai.com (source, opens in a new tab)
  3. Release

    GPT-6 Sol & Luna

    1.05M context · $2 / $10 per 1M tokens

    Two GPT-6 reasoning models. Price shown is GPT-6 Sol (≤272K prompt); GPT-6 Luna costs $0.10 / $0.50.

    developers.openai.com (source, opens in a new tab)
  4. Release

    GPT-6 Astra

    1.05M context · $10 / $50 per 1M tokens

    OpenAI's most capable model for reasoning, coding, computer use and research.

    openai.com (source, opens in a new tab)
  5. Release

    GPT-5.6

    1.05M context · $4 / $20 per 1M tokens

    GPT-5.6 family: Sol (frontier), Terra (balanced) and Luna (high-volume).

    developers.openai.com (source, opens in a new tab)
Full ChatGPT timeline ›

Editor's picks

  1. CVE-2026-129877: DeepAudit exposes LLM API keys and Git tokens to unauthenticated attackers
  2. CVE-2026-129875: REBUILD 4.5.0–4.5.4 password hash disclosure via MCP QueryRecords
  3. South Korea Weighs Nuclear Power to Meet AI Electricity Demand
  4. UK and US Partner to Accelerate Nuclear Fusion with AI
  5. Texas and New Mexico Data Center Emissions Rise With AI Buildout