OpenAI’s GPT-6 Achieves High Accuracy on Legacy Language Migration Benchmarks
Summary
OpenAI has unveiled massive performance gains for its latest GPT-6 models in legacy and low-resource programming languages, hitting a 97.9% accuracy rate on high-effort COBOL migration tests. The release proves that next-generation LLMs can accurately translate archaic code into modern languages without breaking behavioral parity.
What happened
OpenAI released two major technical evaluations: MultiLangBench (which tasks models with building a SQLite-compatible database from scratch in 20 obscure or legacy languages) and a dedicated COBOL migration benchmark. In the latter, GPT-6.1 Sol achieved up to 97.9% accuracy when given high reasoning effort, a historic leap from the 5.4% baseline of GPT-5.6 Sol.
Why it matters now
Trillions of dollars of global financial, aviation, and government infrastructure are trapped in legacy codebases. Proving that LLMs can act as deterministic modernization engines unlocks a massive, highly lucrative new enterprise use-case for AI infrastructure—moving AI from a “coding copilot” to a “tech debt annihilator.”
Who is most affected
Enterprise CTOs, banking infrastructure maintainers, software modernization agencies, and AI coding agent developers who now have an open-source baseline to measure their own tools against.
The under-reported angle
From what I’ve seen covering these releases, mainstream tech coverage is conflating the 97.9% accuracy score with immediate, out-of-the-box production readiness. In reality, scoring high on isolated benchmark functions is vastly different from migrating tangled, undocumented 40-year-old live mainframes, which will require entirely new AI-auditing workflows and regression testing harnesses before deployment.
Deep Dive
Have you ever wondered why most AI coding benchmarks still feel stuck in the Python era? OpenAI’s newest models are breaking out of the Python and JavaScript training-data bubble. By launching MultiLangBench, OpenAI is forcing the AI industry to look beyond heavily saturated coding evaluations like HumanEval. MultiLangBench demands that models construct a complex, SQLite-compatible database engine from scratch across 20 different languages, including low-resource languages like OCaml, Fortran, and Visual Basic. GPT-6 Astra achieved an 87.4% average pass rate across this diverse gauntlet, proving that model intelligence is increasingly generalizing beyond the syntax most common in its pre-training data.
But the true enterprise headline is COBOL. OpenAI’s second evaluation focused exclusively on legacy software migration, testing whether a model can translate COBOL into modern languages (like C++, Java, Rust, or Python) with exact “byte-for-byte parity.” This means the newly generated code must preserve the exact observable behavior of the legacy original. Powered by test-time compute scaling, GPT-6.1 Sol achieved an 86.6% accuracy rate on a “low-effort” setting, and a staggering 97.9% on a “higher-effort” reasoning setting. For context, the previous generation—GPT-5.6 Sol—managed only 5.4% on this exact same task.
That said, while consumer tech outlets are hyping this 97.9% figure as the instant end of legacy software, a critical gap in the narrative is the reality of enterprise deployment. As more cautious voices in the developer ecosystem note, migrating a constrained, synthetic benchmark function is not the same as untangling a massive banking mainframe fraught with decades of undocumented dependencies. OpenAI’s results do not mean an LLM can flawlessly rewrite a bank overnight; rather, they signal that AI translation is now reliable enough to serve as the core engine inside a heavy human-in-the-loop workflow.
To capitalize on this capability, the AI infrastructure ecosystem will need to evolve. The bottleneck for enterprise modernization is no longer the LLM’s ability to understand COBOL, but the organization’s ability to audit the AI’s output. We will likely see a surge in secondary tooling: AI-native test harness generation, automated regression testing, and compliance-focused sandboxing designed specifically to validate AI-generated code migrations before they hit production.
Ultimately, by open-sourcing MultiLangBench, OpenAI has set a trap for its competitors. Standard coding benchmarks are maxed out. The new frontier of LLM evaluation is complex reasoning in environments where human training data is virtually non-existent. This breakthrough heavily implies that OpenAI has figured out how to use synthetic data and advanced test-time reasoning to conquer domains that were previously thought to be immune to AI disruption.
Stakeholders & Impact
|
Stakeholder / Aspect |
Impact |
Insight |
|---|---|---|
|
AI / LLM Providers |
High |
Shifts the battleground from basic code generation to high-stakes enterprise modernization; sets a new evaluation standard for reasoning without massive training data. |
|
Enterprise CTOs & Banks |
High |
Unlocks the potential to automate the migration of decades-old mainframes, but introduces massive new requirements for auditability and regression testing. |
|
DevTools & AI Tooling |
High |
Creates a booming market for secondary infrastructure—specifically automated testing and sandbox environments built to validate AI-translated code. |
|
Regulators & Policy |
Significant |
Code governing financial and government systems must maintain strict compliance; AI-migrated systems will face intense scrutiny for latent, machine-generated vulnerabilities. |
About the analysis
This independent, research-based analysis draws from OpenAI’s official benchmark publications, secondary technology reporting, and developer ecosystem commentary. It is designed for CTOs, engineering managers, and AI infrastructure leaders evaluating the realistic production impact of LLM-driven legacy modernization and the evolving limits of AI coding models.
i10x Perspective
Behavior-preserving code translation is the ultimate test of an LLM’s reasoning capabilities because the output cannot just be plausible—it must be deterministically perfect. OpenAI’s success in low-resource languages proves that test-time compute (high-reasoning effort) can successfully substitute for massive volumes of pre-training data.
Over the next five years, the most valuable enterprise AI deployments won’t just be building new apps from scratch; they will act as an automated immune system, systematically clearing out the trillion-dollar backlog of global technical debt.
