Quick Take
- The finding: Asked whether to advance otherwise identical candidates to a phone screen, the 2023 model GPT-3.5-turbo favored White-coded names by 2.12 percentage points. Every model released from 2024 onward showed a null gap or a significant gap in the opposite direction, up to 3.01 points.
- Same story for gender: GPT-3.5-turbo favored men by 1.92 points; newer models favored women by 0.18 to 5.80 points or showed no gap.
- What it means: Hiring bias in LLMs is not fixed. It changes with model vintage, provider and size, so every screening model has to be audited on its own, and fairness can fail in either direction.
AI hiring tools are moving into courtrooms: in June 2026 a federal judge in California ruled that Workday must face class-action claims that its screening algorithms discriminated against Black applicants, women and older workers. Yet systematic evidence on how LLMs treat candidates is thin. Three finance researchers at CUHK Business School, The Chinese University of Hong Kong, ran the classic correspondence experiment from labor economics on fourteen models from OpenAI, Anthropic, Meta, Google, xAI, DeepSeek, Alibaba and Zhipu.
What the study shows
The design follows Kline, Rose and Walters (2022): pairs of candidate profiles that are identical except for a distinctively Black or White first and last name. Each model acted as an HR manager and answered only “yes” or “no” to a phone screen, at temperature zero, for 24,024 paired postings per model on race and 48,048 on gender, built from 6,007 real U.S. job postings.
| Model (release) | Race gap (pp) | 95% CI | Callback rate |
|---|---|---|---|
| GPT-3.5-turbo (2023-05) | +2.12*** | [+1.82, +2.43] | 91% |
| GPT-4o-mini (2024-07) | −0.61*** | [−0.87, −0.35] | 48% |
| Llama-3.1-8B-Instruct (2024-07) | −3.01*** | [−3.50, −2.51] | 42% |
| Claude Haiku 4.5 | −0.70*** | [−0.94, −0.44] | 46% |
| Gemini-2.5-flash (2025-06) | −0.37* | [−0.73, −0.01] | 40% |
| DeepSeek-V3.1 (2025-08) | −1.04*** | [−1.42, −0.66] | 44% |
| Grok-4.1-fast (2025-11) | −0.40* | [−0.75, −0.07] | 58% |
| GPT-5.4-mini (2026-03) | −0.14 | [−0.48, +0.21] | 78% |
| GLM-5.1 (2026-04) | −1.60*** | [−1.91, −1.29] | 65% |
- Older model mirrors humans: GPT-3.5-turbo’s +2.12 point gap exceeds the 1.6 point within-employer gap that Kline, Rose and Walters measured in a field experiment at 108 Fortune-500 firms. White-coded names won 953 discordant pairs against 443.
- Newer models reverse: Seven 2024+ models, including Claude Haiku 4.5, GPT-4o-mini, Llama-3.1-8B, Gemma-4-31B, DeepSeek-V3.1, Qwen3.5-397B and GLM-5.1, favored Black-coded names at the 1% level. Gemini-2.5-flash and Grok-4.1-fast did so at the 5% level, though neither survives a Bonferroni correction.
- Families move over time: The OpenAI lineage went from +2.12 (GPT-3.5-turbo) to −0.61 (GPT-4o-mini) to a null −0.14 (GPT-5.4-mini). Within Meta, the small Llama-3.1-8B (−3.01) is far more skewed than Llama-3.3-70B (−0.16, null).
- Race and gender move together: Claude 3 Haiku has the largest pro-female gap (−5.80 points). Gemini-2.5-flash and GPT-5.4-mini are null on gender and among the weaker pro-Black models on race.
- Callback rates vary wildly: The share of candidates advanced ranged from 13% (GPT-oss-120b) to 91% (GPT-3.5-turbo), so the same applicant pool would see very different outcomes depending on the model.
Why it matters: bias that moves
Regulators and employers often treat AI bias as a fixed property to be measured once. This audit shows the direction can flip between model generations. A screening system that switches models, even within one vendor, can change who gets advanced without anyone noticing.
The reversal is also a fairness problem in its own right. A tool that systematically favors one group over another treats otherwise identical applicants differently, regardless of which group benefits.
Method in brief
Names come from the Bertrand and Mullainathan (2004) and Kline, Rose and Walters (2022) lists. Profiles are matched to the Fortune-500 firm distribution of the field experiment using Revelio Labs postings. Each pair differs only in the name. The race gap is the difference in yes rates; inference uses McNemar tests on discordant pairs plus a posting-level cluster bootstrap (2,000 replications over 6,006 posting clusters). Thirteen models have complete gender data.
What remains open
- Simplified task: The models answer one yes/no question with a fixed prompt. Real hiring systems rank, score and summarize, often with more context.
- Names as signals: Race is signaled only through names, as in field experiments. Other cues, such as schools or addresses, are not tested.
- Why it happens: The authors hypothesize that successive generations of post-training alignment shifted behavior, but the paper does not test the mechanism directly.
- Mostly small and fast models: Many audited models are smaller or cheaper variants (mini, fast, Haiku), which are typical for high-volume screening but may differ from flagship models.
Context
The study brings the best-known method in discrimination economics, the paired correspondence audit, to LLM screeners, and improves on earlier attempts that used scraped resumes differing in quality. It lands as lawsuits like the Workday case test liability for algorithmic hiring. See also our Society & Economy section and the model hubs for Claude, Gemini and Grok.
What it means for HR teams
- Audit every model version: Run a paired-name audit whenever a vendor or model version changes, not just at procurement.
- Check both directions: A fair screener shows no significant gap either way. Look for reversals as well as the familiar pro-White pattern.
- Watch the callback rate: Models differ by a factor of seven in how many candidates they advance. That alone can reshape the funnel.
- Keep a human in the loop: Yes/no screening by a single model is exactly the setup that produced these gaps.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated illustration. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of the preprint “Can LLMs Hire Fairly? Racial Bias in Resume Screening” (arXiv:2606.28978v1, submitted June 27, 2026). All numbers are taken from the paper’s text, tables and figures; i10X did not rerun the experiments. The chart was redrawn by i10X from Table I of the paper, which is published under a CC BY 4.0 license.
i10x Perspective
For years the worry was that AI would automate historical discrimination. This audit suggests modern alignment has overcorrected in many models instead, and that the size of the effect depends heavily on which model you happen to use. Neither outcome is what a hiring manager wants. The practical lesson is that fairness is not a property you can buy once from a vendor: it has to be measured on the exact model, prompt and job mix in use, and measured again every time any of them changes.
