The economic puzzle behind AI giants’ doomsday warnings

OpinionDeputy business editorSeptember 14, 2026 — 5:00amSeptember 14, 2026 — 5:00amArtificial intelligence researchers are not only racing to develop super-intelligent models, they’re also vying to out-do each other in making ever more apocalyptic predictions.But do all these warnings about the dangers of AI from big AI firms and their staff strike you as a bit rich?Researchers from Anthropic said the company was worried about the threat AI posed to humanity.AP Photo/Patrick SisonLast week the top scientist at OpenAI warned “no one is prepared for the consequences of a continued rapid rise in machine intelligence,” then days later Anthropic researchers claimed the AI giant was worried the technology could wipe out humanity.These are merely the latest cases of AI doom-saying. We’ve also been told by AI firms that there should be a global freeze on development of the most powerful systems, or that some of their models are too dangerous to publicly release.While some of these claims are probably just hype, many of the people making these warnings no doubt mean it.Even so, there is a glaring tension: the people who are raising the alarm are often employed by (or have recently quit) the very companies that are driving the AI craze.If AI firms hold genuine concerns about the future of humanity, thanks to a technology they are rushing to develop, surely they would be the best people to do something about it by tapping on the brakes, right?But we know the firms won’t voluntarily do this, because there’s just too much money to be made from the AI mania. Anthropic is reportedly eyeing a stockmarket float next month, and there have been unverified reports of a $US2 trillion ($2.8 trillion) valuation. There have also been unconfirmed reports OpenAI is also keen for a $US1 trillion valuation when it ultimately floats.Economics and other social sciences have a phrase for describing this sort of puzzle – where businesses often do things even if they know it’s self-destructive. It’s known as a “collective action problem.” It’s a useful way to look at the AI mania and what might be done to contain the risks, if policymakers can act quickly and decisively enough.Collective action problems are situations where businesses would be better off working together – such as agreeing to pause the development of these high-risk AI models – but no one wants to do so out of fear of losing the competitive race. They’re a downside of competition, and we’ve seen them appear in all sorts of situations before.During the banking royal commission in 2018, for example, there were cases where banks continued to engage in dubious practices because they were afraid of losing business to rivals. The solution was for the government to step in and ban the dodgy practice, to save the firms from themselves.In the AI race, greater regulation would also surely help to deal with some of the more catastrophic (if unlikely) risks AI insiders are warning of. But it’s much harder to put the brakes on AI development than it is to ban banks from selling certain products.One reason is harder is that it’s not only AI companies that are competing with each other, but also nations. So even if one country were willing to rein in its AI sector, it might not actually do so if rival nations don’t also agree to tap the brakes.It’s a diabolically tricky problem. Technology billionaire Bill Gates has said there will need to be some sort of international organisation for “dealing with AI”, perhaps similar to the regime for inspecting nuclear weapons, or regulating international aviation.Bill Gates has drawn comparisons between regulating AI and the regime for inspecting nuclear weapons.Getty Images for Bloomberg PhilanthropiesIndependent economist Chris Richardson says the type of co-operation needed to deal with AI risks could be “spectacularly hard.” He compares it to dealing with climate change – something the world’s been grappling with for decades. Worryingly, AI is moving much faster than climate change.“Most of the nations and businesses regard themselves as highly competitive with each other,” Richardson says. “To solve a global problem, you need some degree of global cooperation, and we are just not good at that.”UNSW Scientia Professor of economics Richard Holden says there are examples of ways to address a collective action problem, and he believes much of this could be done in the US, because it’s leading on AI frontier model development.While he’s sceptical of some of the AI researchers’ more outlandish claims, he says policymakers can respond to collective action problems by either regulating, or encouraging co-operation to head off the risks.“It could be ‘stop racing too fast and releasing things before they’ve been vetted properly,’ or it could be encouraging people to share information with each other, share information with third parties,” Holden says.AI firms have every incentive to tell dramatic stories about how amazing their product is, and we need to be sceptical about wild claims about the risks to humanity from their technology. Some of this is self-serving hype.But if there’s even slim chance these doomsday warnings are correct, then there should be greater regulating or vetting of the most advanced AI models, to limit the risk of some unintended disaster. Economics and common sense tell us the companies can’t be counted on to slow down by themselves.The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning.Clancy Yeates is deputy business editor. He has covered banking and financial services, and was previously national business correspondent in the Canberra bureau.Connect via X or email.From our partners