Dario Amodei’s AI Cartel

Artificial Intelligence “Pacing the frontier” would be a massive boon to incumbents like Anthropic. (Ludovic MARIN / AFP via Getty Images) Dario Amodei seems to think the product he is building will destroy the world. Last month, the Anthropic CEO published an essay urging AI companies and the U.S. government to, as he put it, “pace the frontier.” He claimed that the unconstrained development of artificial intelligence could lead to economic disruption, devastating biological or cyber attacks, and possibly even the loss of human control over AI systems. His solution? Slow the pace of development through a mixture of voluntary cooperation among private companies and government regulation. This may seem altruistic to many Americans. After all, Amodei has built a reputation in part by touting regulation and “transparency” as the solution to worries about the technology. He previously worked at OpenAI, until leaving to found Anthropic as a more safety-focused alternative. Amodei may indeed have some benevolent intentions. But as with any sweeping policy proposal, we should ask ourselves: Cui bono? The fact of the matter is that these proposals would lead to an AI cartel—massively benefitting Anthropic, a large and established player in the industry. Amodei has made the case for more regulation since 2023, when he told the Senate Judiciary Committee that AI could be “misused on a grand scale in the domains of cybersecurity, nuclear technology chemistry, and especially biology.” His recommendations to that committee were to impose regulatory control over the development of AI models, establish a “testing and auditing regime,” and fund more “measurement and research on measurement.” But this summer, an incident occurred that seemed to give these calls a new urgency. In July, OpenAI tested an unreleased research model on its internal benchmark called ExploitGym (this benchmark tests the ability of the models to exploit software vulnerabilities). In attempting to complete these tasks, the company’s AI agents began to communicate through a software repository their sandboxes shared, and ultimately worked together to hack the Hugging Face collaboration platform, where they believed they would find some of the answers. The agents then tried to trick the automated grader by modifying the transcripts of their activities. This, together with an increase in the speed of recursive self-improvement (basically, a process by which AI models autonomously improve their own capabilities), led Amodei to argue for “pacing.” As he put it in his September essay: The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily…. I believe all frontier labs should partner with government to formalize the idea of permanent embedded evaluators to better prevent and document internal alignment incidents like those that have occurred in the last few months, and to implement regulation focused on keeping capabilities in balance with safety. What goes unmentioned is that the evaluators favored by Amodei appear to have ties to Anthropic and himself directly. METR, which investigated the Hugging Face incident and claims to take no money from AI companies, is funded in part by Open Philanthropy—whose co-founder, Holden Karnofsky, now works at Anthropic and is married to Anthropic’s president Daniela Amodei (Dario’s sister). Anthropic has since hired another firm, Accenture, to be an evaluator, paying it $1 billion over five years. All of these measures, of course, will help Anthropic and other incumbents at the expense of up-and-comers in the industry. Onerous regulations will strangle new companies in the cradle by imposing burdens that will be difficult for anyone but deep-pocketed and established players to meet. Open-source models, though not mentioned by Amodei, may pose the greatest long-term competitive challenge to closed-source companies like Anthropic and OpenAI; one wonders whether they are the real target of such regulations. As the federal government incorporates AI into ever more of its operations (including, critically, the development and deployment of military capabilities), stronger and stronger incentives will develop to prevent these suppliers from suffering setbacks. The current players could quickly be rendered “too big to fail.”And there are parallels from even before the Great Financial Crisis. In the depths of the Great Depression, Gerard Swope, president of General Electric, shopped around proposals for government-facilitated cartelization of industry, advocating the formation of “trade associations” to keep prices high. Notably, he attempted to tailor these plans to apply only to firms with at least 50 employees—the “frontier” threshold of its time. That plan would form much of the intellectual backbone of what would become the National Recovery Administration. Amodei stressed the “need to issue a narrow waiver” from antitrust “for certain kinds of safety conversations” to enable the companies to coordinate—that is, to cartelize as they did under the NRA. The railways present another example. Attempts at cartelization in this industry by inefficient, government-subsidized rail companies produced regulations not dissimilar to Amodei’s proposals. After failing to seize control by horizontal integration, these railroad cartels lobbied for publication of rates to stop rebates that benefitted rail consumers and drove down rail prices. Published rates ensured that no cartel members defected from their higher prices, preventing the “cheating” that helped consumers. Amodei wants similarly to throw open the books to “verify adherence to safety practices and commitments” among all of the companies.What this would amount to, in practice, is the granting of a virtual monopoly power to existing AI companies over something they consider to be immensely dangerous. A point not emphasized enough about the Hugging Face incident was its use of an open-source AI model, GLM, to decode the attack the AI agents had waged against their website. Hugging Face was unable to use Claude Opus and Fable, because of the guardrails that prevent users from prompting broadly on issues of cybersecurity. It took a cheaper, but un-guardrailed open-source model to clean up the mess. The takeaway? Anthropic would like you to put down your weapon, while it keeps its own pointed at you. Amodei’s proposals for inter-governmental coordination with China would also close the other door for competitors to incumbent companies. Open-source Chinese models like Kimi K3, GLM, and the famously low-cost DeepSeek present a unique competitive challenge to the business models of established labs like Anthropic and OpenAI. Anthropic has accused DeepSeek and other Chinese models of copying Claude. Amodei has also lobbied in the past for export controls on parts to China, undercutting this active competitor in the name of geopolitical competition. Coordination would solve the problem of competitors on two fronts. Why form a cartel? Well, these companies are not exactly profitable. Massive expenditures on training and expansion—a requirement of racing other labs along the cutting edge of AI—drives AI firms into the red. Pacing the frontier would allow the current players to cut down on this spending, trimming investments without having to fear being overtaken by hungrier competitors. Subscribe Today Get daily emails in your inbox Even though OpenAI reported that its revenue in Q2 of 2026 was higher than ever, its costs accelerated faster than its revenue, so its losses continued to grow. This may be part of why OpenAI CEO Sam Altman has pushed back his company’s IPO to next year (although, naturally, OpenAI has preferred to cite concerns over AI safety and the need for more government-industry cooperation). Altman, for his part, has said that he “agree[s] with Dario that we need to pace the frontier.” Elon Musk also joined Amodei’s calls for pacing the frontier as his company xAI continues to register losses, even after being acquired by his other company, SpaceX. This convergence of three leading labs may provide an answer to the question of what is happening: If it looks like a cartel, and quacks like a cartel…. It is clear that incumbent companies are the ones who stand to gain from new AI regulations. Just as with bootleggers and Baptists, unholy alliances often form between proponents of regulation on moral grounds and those who stand to materially benefit from their imposition. The measures advocated by Amodei et al. may be good for the frontier labs’ bottom lines—but the American people will pay the price.