Anthropic’s Opus language problems may be creating a hidden cost for AI coding

Anthropic’s Opus language problems may be creating a hidden cost for AI coding

Language problems that require repeated correction could create productivity drag, operational risks, and hidden costs for enterprise development teams, analysts say. AI coding assistants are supposed to reduce the work required to turn a developer’s intent into working software. But some users of Anthropic’s Opus 4.8 and Opus 5 models say they are having to spend additional time, prompts, and tokens correcting the models’ language, sometimes even routing their output through cheaper AI models to make it usable. In a detailed GitHub issue, Peter Bower, founder and CEO of London-based tech startup SpaceCell, said that Opus 4.8’s tendency to use confusing or invented terminology was creating extra work in software development workflows, particularly when generating code documentation. That was despite being explicitly and repeatedly prompted to avoid certain terms and use specified alternatives, Bower wrote, adding that the model continued to introduce the unwanted terms, forcing repeated cleanup passes, including through cheaper Sonnet or Haiku models, to make the documentation “sane and presentable.” Those additional passes, he further said, were pushing token costs up to two times higher than they otherwise would have been. Bower’s issue, which was posted last month, has since received nearly 265 acknowledgements, which could indicate that several other users have faced an issue with Opus 4.8’s language coherence one way or the other. Some even commented on having faced a similar issue. Bower himself also references a ClaudeAI subreddit in his issue, which points to Opus 4.8’s language incoherence. That, too, received a significant number of upvotes, which are Reddit’s equivalent of a thumbs-up that is often used on social media to indicate approval or support for a post or comment. Another subreddit thread points to a similar issue with the Opus 5, with users reporting the model’s tendency to produce confusing, hard-to-parse output, and it received nearly twice as many upvotes. Why unclear AI output could slow software development For enterprise development teams, the persistent nature of the reported issue with the Opus models could result in significant productivity drag, analysts say. “Repeated correction cycles can erode productivity when developers spend enough time reviewing, redirecting and repairing AI output. That offsets the time saved by generating code via a coding assistant or any other tasks,” said Abhishek Satapathy, principal analyst at Avasant. That erosion in productivity, according to Advait Patel, senior site reliability engineer (SRE) at Broadcom, is also linked to the operational aspects of the software development lifecycle (SDLC) as unclear AI-generated prose could affect design documentation, runbooks, architecture decision records (ADRs) and incident writeups. “A runbook written in a style that engineers find difficult or unpleasant to read, for example, could become a problem during an incident, when teams need to quickly understand and act on the information in front of them,” Patel said. Code review, Patel added, presents another potential problem due to unclear prose: “Overly padded or confusing pull request descriptions are likely to be skimmed rather than carefully reviewed, increasing the risk of important details or potential defects being missed.” Unclear output could have repercussions on cost The implications of unclear prose extend to costs as well. That is because the price enterprises pay for an AI coding tool does not necessarily reflect the cost of getting usable output from it, said Bhupendra Chopra, chief revenue officer at IT consulting firm Kanerika. If developers have to make repeated passes to correct, rewrite, or review a response, or route it through another model, then those additional steps become part of the overall cost of completing the task, including human review time, Chopra added. And most enterprises, according to Patel, often don’t realize this calculus because all of this “is packed into a single line item” in their coding agent bill. That hidden cost could also have implications for Anthropic’s ability to retain developers. “Switching coding assistants or underlying models have become relatively easy for development teams, particularly as coding platforms increasingly support models from multiple providers, though enterprises are likely to encounter sunk cost in config, hooks and MCP setup. But the code doesn’t move, the repos don’t move, and thus no migration plan is needed,” Patel said. “That’s a genuine commercial risk for any model vendor. Low switching cost means goodwill is your only lock-in, and readability complaints erode goodwill fast because people hit them daily,” Patel noted. Prompt workarounds may not be enough However, Anthropic has not yet responded to Bower’s GitHub issue, which also outlines the changes he believes the company should make to address the problem. The startup founder has called for Anthropic to tweak the model’s default writing style to be closer to “a technical white paper or a good Stack Overflow answer”, which is “plain, declarative and direct”. He also called for the model to be less verbose while strongly adhering to instructions set in CLAUDE.md and repeated during a conversation, arguing that those instructions should persist rather than gradually being overridden by the model’s default communication style. In the meantime, Patel, who said he has faced similar model drift at work, particularly while working with repositories involving a Jenkins, Python, Terraform, GKE, and Helm stack, pointed to a fix he and his team use when generating documentation and pull request summaries. Rather than broadly asking Claude to be concise, his team uses explicit rules in project configuration to ban specific phrasings, because asking for conciseness can sometimes make the output shorter but more cryptic, Patel said. However, Patel cautioned that relying simply on prompt-level workarounds may not be enough for enterprises because model behavior can change over time. “Model behavior is a moving target,” Patel said. “A version bump can change output register without you deploying anything, and nothing in your pipeline alerts on it.” That means CIOs and engineering leaders should treat changes in model behavior as something that needs to be tested and monitored continuously. “Pin model versions for anything in a pipeline instead of tracking latest. Keep a small eval set of your own real tasks and rerun it on every model change. Track rejection and rework rate, that’s your early warning. And don’t let thirty teams each invent their own undocumented prompt workarounds,” Patel advised.

Original Source

Read the full article at Infoworld →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.