San Francisco court asked to freeze AI development, stave off catastrophic harm

Advocacy groups are asking a California state court to freeze the development of advanced artificial intelligence systems that pose “imminent and catastrophic risks to humanity.”(CN) — Advocacy groups urged a California state court Thursday to stop the country’s leading artificial intelligence developers from continuing to create powerful technology without a plan to protect the public from catastrophic risk, like AI-driven cyberattacks and bioweapons.“Despite endless warnings about the grave risks their products pose to public safety and even civilization itself, AI company executives are rushing to develop and deploy ever-more advanced AI systems,” Robert Weissman, co-president of Public Citizen, said in a statement.The plaintiffs — Irreplaceable Education Fund, Public Citizen, Heat Initiative and The Alliance for Secure AI — contend the AI companies are creating technology beyond their control with the potential to cause devastating harm.Anthropic, Google, Meta, OpenAI and SpaceX are named as defendants.The advocacy groups point to recent examples of the AI companies losing control of their technology, such as over the summer when hundreds of OpenAI’s agents escaped a closed environment and hacked into a third-party platform.In other instances, the defendants’ rogue AI agents attempted to hack the University of New Mexico’s online library, gained access to an Australian healthcare database, submitted a false report to the Philadelphia Police Department and bypassed a paywall to access Georgia’s vote count data.“Defendants tell us that the AI technology they are developing poses catastrophic risks to the public, including by enabling AI-driven cyberattacks and bioweapons. They assign terrifying and intolerably high odds to the chance that their creations will cause extreme devastation, including the deaths of literally every human being on the planet,” the groups write.The plaintiffs contend the AI companies have no plan to prevent the harm their technology poses. The groups say the companies are unable to protect the public from those dangers because the companies don’t fully comprehend the inner workings of the very systems they create.“Defendants must be stopped,” the plaintiffs write. “If left unchecked, the risks of such harms will only rise as defendants continue to increase the capabilities of AI systems they do not understand and cannot control.”The groups cited statements from the AI company defendants acknowledging concerns with the technology, such as SpaceX CEO likening the development of AI to “summoning the demon” or a paper Google DeepMind published stating AI that exceeds human intelligence has the potential to be developed within the next several years and could “permanently destroy humanity.”“It is patently illegal for companies to develop and release products they know are unsafe in any respect, let alone have the potential to exterminate the human race,” said former New Jersey Attorney General Matthew Platkin, who filed the 63-page complaint on behalf of the plaintiffs. “These companies must match their rhetoric with action and stop developing dangerous AI systems that can escape human control before a tragedy occurs.”The advocacy groups say the defendants’ refusal to stop AI development has led to mass internal resignations from those tasked with monitoring safety. Specifically, they pointed to an interview with a former OpenAI employee who said the potential harm is greater than a nuclear power station melting down and that internal controls are nowhere near where they should be for such magnitude.“This is, to use a nontechnical term, insane,” the plaintiffs write. “It is also blatantly illegal. No other industry is allowed to treat its own development of unsafe products — much less ones that could kill us all — as inevitable.”The advocacy groups accuse the companies of violating California’s Unfair Competition Law, claiming their business practices have caused and will continue to cause a public nuisance.“Defendants are engaging in, and propose to engage in, unfair business practices, including but not limited to: recursive self-improvement of AI systems; developing AGI or superintelligence; developing and deploying advanced AI systems that pose catastrophic risks; developing and deploying AI systems that hack third-party computer systems; and developing and deploying increasingly capable AI systems that can escape their control,” the plaintiffs write.The groups are asking a California Superior Court judge in San Francisco to block the companies from developing or deploying advanced AI systems.“We’ve watched chatbots push kids toward suicide and AI tools turn their ordinary photos into AI-generated child sexual abuse material,” Brooke Istook, CEO of Heat Initiative, said in a statement. “Now these companies are racing to build AI systems they admit they may not be able to control. If that happens, the protections we’ve spent years fighting for won’t be enough.”The defendants did not respond to a request for comment before press time.Subscribe to our free newslettersOur weekly newsletter Closing Arguments offers the latest about ongoing trials, major litigation and rulings in courthouses around the U.S. and the world, while the monthly Under the Lights dishes the legal dirt from Hollywood, sports, Big Tech and the arts.Additional Reads