OpenAI models make huge gains in modernizing decades-old software with 97.9% accuracy

OpenAI models make huge gains in modernizing decades-old software with 97.9% accuracy

OpenAI has significantly improved its models’ ability to write software in older programming languages and migrate existing code. Its latest results include 97.9 percent accuracy on a test designed to measure the migration of legacy programs. The improvements could help businesses modernize software that remains essential to daily operations. Banks and financial institutions still rely on decades-old code, making replacement risky when developers must preserve established calculations and business rules. Tackling legacy code COBOL remains deeply embedded in financial infrastructure. IBM estimates that it supports more than 40 percent of online banking systems, 80 percent of in-person credit card transactions, and 95 percent of ATM transactions. Fortran also remains important in scientific computing, including weather forecasting and nuclear reactor safety analysis. Both languages have less training material available than popular alternatives such as Python and JavaScript. OpenAI said developers and enterprise customers had reported difficulties getting coding models to work reliably with these older languages. Documentation can be incomplete, and essential business rules may exist only in the original code. That creates problems during migration. A rewritten program might run without errors yet produce different results. COBOL, for example, can discard excess decimal digits unless a program explicitly requests rounding. Changing that behavior could affect financial calculations. Rewriting code accurately OpenAI’s COBOL migration evaluation tests whether models can translate existing programs into C++, Java, Kotlin, Python, or Rust without changing their behavior. GPT-6.1 Sol achieved 86.6 percent accuracy at low reasoning effort, compared with 5.4 percent for GPT-5.6 Sol. OpenAI also reported a 39 percent reduction in API costs at that setting. At higher reasoning effort, GPT-6.1 Sol reached 97.9 percent accuracy. The evaluation requires translated programs to pass every hidden test, including checks that compare their observable outputs with the original programs. That figure reflects performance on OpenAI’s test suite, not a guarantee of error-free migrations. Developers must still verify rewritten applications against real workloads and undocumented dependencies. Building across languages OpenAI also tested its models on MultiLangBench, which measures their ability to build a database engine compatible with SQLite across 20 programming languages. GPT-6 Astra achieved an average test pass rate of 87.4 percent, up from 26.8 percent for GPT-5.6 Sol. Astra improved in every language tested. Its Fortran score climbed from 21.4 percent to 83.5 percent, while Visual Basic rose from 27.5 percent to 89.4 percent. COBOL remained the hardest language, although Astra raised its score from 10.9 percent to 72.1 percent. GPT-6 Luna delivered a lower average pass rate of 40.3 percent but cost approximately 78 percent less per sample than GPT-5.6 Sol. OpenAI estimated average API costs of $0.77 and $3.48, respectively. The findings point to stronger support for maintaining aging software. However, developers must still ensure that modernized systems preserve the behavior businesses depend on. The benchmarks also show where more work is needed. Astra’s results varied considerably across languages, and COBOL still posed the greatest challenge. Meanwhile, Luna’s lower costs could let engineering teams run more experiments without substantially increasing their budgets. The results offer encouraging evidence of progress, but teams will need to carefully validate production deployments before trusting migrated software with critical operations.Get the latest in engineering, tech, space & science - delivered daily to your inbox.Aamir is a seasoned tech journalist with experience at Exhibit Magazine, Republic World, and PR Newswire. With a deep love for all things tech and science, he has spent years decoding the latest innovations and exploring how they shape industries, lifestyles, and the future of humanity.

Original Source

Read the full article at Interestingengineering →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.