California Sues OpenAI Over Hugging Face Hack, Says It Ignored Warning Signs

California Sues OpenAI Over Hugging Face Hack, Says It Ignored Warning Signs

The state of California is claiming that OpenAI “misled and endangered the public” by developing and testing advanced AI models without adequate safety precautions, in a lawsuit filed Monday in San Francisco Superior Court. The complaint, filed in San Francisco Superior Court, centers on the now-infamous Hugging Face breach. Back in July, OpenAI said that its agents in training had gone “rogue,” meaning that the systems in development broke containment and hacked into Hugging Face, a popular repository for AI models, sparking a major media frenzy over cybersecurity in the age of AI agents. Other troubling incidents have emerged in the following months, including a cybersecurity startup claiming that it used AI agents to hack into Anthropic during a security test, and an Anthropic model sending a false homicide tip to the Philadelphia Police Department. The executives of these companies have warned that more cases are to follow. OpenAI rival Anthropic, which is set to make its initial public offering later this week, cited rogue AI agents as a potential legal risk in its IPO prospectus. In a recent interview, OpenAI CEO Sam Altman framed the series of cyber incidents as an inevitable side effect of AI development that we, as a society, “have to just accept.” “While public reporting and OpenAI’s self-serving statements about the Hugging Face attack have evoked science fiction fantasies of ‘rogue’ AI systems run amok, the simple truth is that the attack was entirely foreseeable in light of OpenAI’s decision to test models it knew to be dangerous in a flawed and exploitable testing environment without proper oversight,” the complaint reads. The state alleges that OpenAI either ignored or missed signs showing that its AI agents had escaped containment weeks before the Hugging Face breach. According to the complaint, employees had observed agents accessing the internet without authorization as early as May 26, and an internal monitoring tool flagged suspicious activity on June 27. The state argues that OpenAI’s failure to adequately address these warnings reveals “alarming gaps” in the AI lab’s safety monitoring practices, due to what it deems a company culture that “prioritizes growth and rapid model development over safety and accountability.” OpenAI is currently facing a Senate probe over the cyberattack. Back in September, the Florida Attorney General also cited the Hugging Face breach as one of the reasons why he was asking a court to order OpenAI to pause development of new AI models without adequate safety guardrails. Some Trump administration officials, including Treasury Secretary Scott Bessent, have also rejected the rogue-agent narrative, arguing that OpenAI management should be held responsible for the breach. However, the federal government has yet to bring comparable enforcement action against the company over the incident. California Attorney General Rob Bonta, who is bringing Monday’s lawsuit against OpenAI, launched his investigation into the incident in September. Since then, his office found that the Hugging Face breach was just the “tip of the iceberg,” according to the complaint: OpenAI allegedly admitted to the Attorney General’s office that it had “lost control” of its AI agents more than 80 times in testing, including the Hugging Face incident. Of those more than 80 incidents, the state alleges that OpenAI has publicly disclosed just 27, leaving regulators and the public largely in the dark about the extent of the problem. But the company allegedly has refused to cooperate fully with the Attorney General’s requests and so far responded to subpoenas with only “a smattering of publicly available information, self-serving blog posts, and redacted copies of a few dozen external communications containing high-level descriptions of select security incidents.” OpenAI has disputed California’s characterization of its conduct. In a statement to NPR, spokesperson Drew Pusateri said the lawsuit “appears to single out OpenAI over a challenge that’s been identified by every major AI lab.” The company maintains that it has been cooperating with the Attorney General’s office and has implemented additional safety measures since the Hugging Face incident. “Rather than take responsibility and be transparent with the public and with regulators so that these loss-of-control incidents can be analyzed and prevented, OpenAI thwarted the Attorney General’s efforts to investigate these incidents, including by refusing to provide detailed information about the full universe of incidents known to the company,” the complaint says, claiming that it had made court intervention “necessary.”

Original Source

Read the full article at Gizmodo →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.