Quick Take
- The finding: In two preregistered experiments with 2,193 US adults, AI chatbots reduced conspiracy beliefs mainly by supplying relevant facts and counterarguments. When the AI was told to persuade without evidence, most of the effect disappeared.
- What did not matter much: Telling people the AI wanted to change their minds, framing the chat as a debate, telling the AI to inform rather than persuade or halving its answers did not detectably weaken the effect.
- The surprise: A single tailored AI message kept an estimated 79% of the effect of a full conversation. Low trust in AI and beliefs shared by friends and family did not block the effect either.
In 2024, Thomas Costello, Gordon Pennycook and David Rand showed in Science that short conversations with GPT-4 can durably reduce belief in conspiracy theories. The open question was why: deference to a seemingly neutral machine, or the chatbot’s warm, patient style? A new preprint from the same team at Carnegie Mellon University, Cornell University and MIT, “Just the Facts: How Dialogues with AI Reduce Conspiracy Beliefs”, removes possible ingredients one at a time. The answer is in the title.
What the study shows
Participants described a conspiracy theory they found credible, an AI model turned it into a belief statement, and they rated it from 0 (definitely false) to 100 (definitely true). Only ratings of 50 or higher counted. After the AI interaction, they rated it again.
- Baseline replicated: In Study 1 (N = 1,290, GPT-4), the original dialogue lowered belief from 80.5 to 69.6 points, a mean drop of 10.9 points. In Study 2 (N = 903, GPT-5.2), belief fell from 80.5 to 65.9, a drop of 14.6 points, while an unrelated control chat (for example about cats versus dogs) moved it by only 1.4 points.
- Facts are the active ingredient: When the AI was told to persuade without evidence, counterarguments or reasoning (“No Facts”), the drop shrank to 3.81 points. A Bayes factor of 11.43 indicates strong evidence of a difference from the baseline.
- Most other changes made no detectable difference: Telling participants the AI would try to persuade them, framing the chat as a debate, telling the AI to inform rather than persuade, or roughly halving its answers (588 vs. 1,226 words per conversation) all produced drops between 9.10 and 13.83 points.
- Beliefs changed sides: In Study 2, 23.1% of baseline participants who started above the midpoint ended below it, compared with 1.5% in the control group. In Study 1 the figure was 16.5% for the baseline and 5.7% for No Facts.
- One message is almost enough: A single tailored AI message lowered belief by 11.76 points and kept an estimated 79% of the dialogue’s effect relative to control.
| Condition (Study 1) | What changed | n | Mean belief drop (points) |
|---|---|---|---|
| Baseline | Original dialogue from the 2024 study | 333 | 10.88 |
| Just-the-Facts | AI told to inform, not persuade | 127 | 11.62 |
| Concise | Answers as short as possible | 137 | 13.83 |
| Adversarial | Chat framed as a debate | 131 | 11.85 |
| Overt | Users told the AI will try to persuade them | 132 | 9.10 |
| No Facts | AI persuades without evidence or arguments | 129 | 3.81 |
Content analyses point the same way. Without evidence, the model leaned on stories and empathy, and only 9.9% of its claims were specific empirical facts, compared with 44.5% to 53.4% in most other conditions. Among the strategy dimensions, “Facts & Evidence” and “Sources & Experts” were linked to larger belief change; rapport, stories and appeals about harms were not.
Why it matters: The evidence behind AI debunking is unusually strong
This follow-up was designed to try to break a high-profile result. Each condition removes one candidate explanation, and both studies were preregistered. Because many results are nulls, the main text relies on Bayesian models, which can quantify evidence for no difference; the preregistered frequentist tests are in the supplement. Study 2 adds a randomized control chat to rule out regression to the mean and tests a newer model, GPT-5.2.
The team also checked the measurement itself. Rating belief before people elaborated on their evidence, or rating a statement stripped of that evidence, did not change the result. Counting dropouts as “no change” left the Study 2 effects intact.
Method in brief
- Participants: US adults via CloudResearch Connect. Study 1 kept 1,290 of 3,988 people after preregistered exclusions; Study 2 kept 903 of 2,082 and was sampled to balance Democrats and Republicans.
- Models: GPT-4 ran the dialogues in Study 1 and GPT-5.2 in Study 2; GPT-4-family models restated beliefs and classified them as conspiracies.
- Design: Study 1 had eight conditions compared with the baseline dialogue. Study 2 compared the baseline dialogue, a single tailored message and an unrelated control chat.
- Content coding: GPT-4o rated 16 persuasion strategies; GPT-4.1 extracted about 58,000 claim instances, which Gemini 3 Flash classified.
- Moderators: Trust in AI was tested in a pooled sample of 2,594 people from three experiments. Each point of higher trust added 2.13 points of belief reduction, but people 1 SD below average trust still moved 10.7 points.
What remains open
- Only large differences detectable: Each null contrast in Study 1 could only detect differences of about 6 to 7 points (53–63% of the baseline effect). Smaller contributions from dialogue or trust in AI cannot be ruled out.
- “Facts” is a bundle: The No Facts prompt removes specific facts, source citations and precise claims all at once, so the study cannot say which of them matters most.
- Observational content analysis: The strategy and claim analyses are post hoc, correlational and coded by AI models.
- Sample: Self-selected online participants, not a representative US sample.
- No new follow-up: Belief was measured right after the chat; durability evidence comes from the 2024 study.
- Reproducibility and funding: The authors say data and code will be deposited with persistent identifiers when the paper is accepted. The preprint has no funding or conflict-of-interest statement. The authors disclose that they used Claude to write and check analysis code and to edit the manuscript.
Context: From DebunkBot to a mechanism
The paper builds on Costello et al. (2024, Science), whose effect persisted at a 2-month follow-up. In the same paradigm, believing one was talking to a human expert produced comparable reductions (Boissin et al., 2025). Other work agrees: removing factual claims sharply reduced AI canvassers’ persuasiveness in the 2025 Canadian and Polish elections, and fact-heavy persuasion was most effective on UK political issues. i10X has also covered how AI literacy warnings cut political persuasion and how bias disclaimers work unevenly across party lines.
What it means for platforms, educators and AI teams
- Lead with evidence: For debunking or fact-check assistants, accurate counterevidence tailored to the person’s reasons matters more than empathy scripts or a friendly persona.
- Short can work: Halved answers kept the effect, and a single tailored message kept most of it, which makes deployment cheaper.
- Transparency is affordable: Disclosing the persuasive goal did not detectably weaken the effect.
- The same force cuts both ways: If fact-dense arguments are this persuasive, the accuracy of the model’s facts becomes critical. Model providers like OpenAI, whose models were used here, and teams building on them need strong fact-checking and misuse safeguards.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint, including the supplement (arXiv:2611.05572v1, submitted November 3, 2026). All numbers come from the paper; i10X did not rerun the experiments. The paper is licensed CC BY 4.0. The chart was redrawn by i10X from the reported Study 1 reductions (main text and Table S7) for readability. AI tools assisted with research and drafting; the selection, fact-checking and final editing were done by the i10X editorial team. More research coverage is in our Society & Economy studies.
i10x Perspective
Conspiracy believers are often written off as unreachable by facts. This paper suggests the problem is that nobody delivers the right facts at the right moment, which chatbots do well. That is encouraging, and it raises the bar: a system this persuasive with true facts must be held to a very high standard of accuracy.
