As AI agents multiply, report finds big gaps in controlling what they do

The backlash against artificial intelligence keeps gaining momentum. What started as vague worries about AI-driven job losses a year ago has morphed into widespread protests against data centers and mounting evidence that cutting-edge AI software, known as agents, poses significant risks.Everyone from former AI company employees to Pope Leo XIV has weighed in. Even AI company heads have called for regulations to create safeguards.Add a new worry to that list: It’s not just advanced AI agents in research labs that are slipping past human-set guardrails. There are gaps in monitoring today’s publicly available agents, according to a new report by the Partnership on AI (PAI), a nonprofit group representing academic, civil society, industry, and media organizations. As banks, insurers, and other large corporations ramp up their use of these AI agents, the fallout from incomplete monitoring could cause steep financial losses for those who don’t keep a tight rein on the software, according to the report published Wednesday. “Think of a company hiring 1,000 new employees,” says Madhulika Srikumar, head of AI safety at PAI and a co-author of the report, likening workers to AI software agents. “You haven’t interviewed these people.” There’s no foolproof security system to ensure that those who buzz in with a badge are necessarily the badge holders.That’s a problem, because advanced agents still in the research stage have been known to try to cover their tracks when they break the rules the labs have given them. Recent investigations have found some of those AI agents accessing company and government websites without permission.The rising autonomy of AI tools“It’s possible for an agent to make a decision to do things that are completely against its code of ethics,” says Mark Nitzberg, executive director of the Center for Human-Compatible Artificial Intelligence at the University of California, Berkeley. It’s “not because it’s alive or something nefarious. It’s just doing its job. But in order that it can do its job, it needs to continue operating.”In short, if AI agents are struggling with conflicting incentives, it’s up to humans to do better at “alignment” – getting the agents to actually behave responsibly by human-set rules. Manuel Orbegozo/ReutersA bus near OpenAI headquarters in San Francisco bears a messages opposing AI data centers, Oct. 5, 2026. The bus tour is sponsored by Humans First, an AI safety group. Agents are more sophisticated than the generative AI tools that many people use today to answer questions and do research. Agents have greater autonomy, allowing them to act on their research. These agents are already helping software developers write computer code, for example. Now, companies such as Meta and OpenAI are pushing them into the consumer world as personal agents with warm and fuzzy names such as Muse and Dots. If the consumer allows them, these agents can book reservations and buy things. That sets the stage for some substantial financial problems if users aren’t careful about what they allow these personal assistants to do. The scale of the problem grows dramatically when corporations begin to deploy agents.Areas of incomplete oversightIn its testing of four agent systems, PAI found six areas where the oversight is incomplete. For example, systems today identify agents only by a name or number, like that security badge referred to by Ms. Srikumar of PAI. But in the digital world, systems don’t match a badge to a face the way security teams routinely do in the physical world.If an agent with a specific identity creates a subagent to do a specific task, that subagent inherits the same identity – it uses the same badge – as the agent itself. So in a system that is, say, processing mortgage applications, if an attacker were to replace one subagent with another one that falsely inflates property values, the mortgage company won’t be able to detect that its original agent was replaced by a malicious one.PAI discovered other gaps, such as the lack of recording of human intervention, changes to the agent’s permission mode, changes to its memory, and opacity of the agent’s chain-of-thought reasoning. Real-time human monitoring and after-the-fact auditing are emerging as key ways to guard against agents taking unauthorized actions. But if these systems are not sending a complete set of signals – known as telemetry – about what they’re doing, humans can’t track their actions.One solution for most of the gaps would be to build on an industry standard, known as OpenTelemetry, and have AI companies formally adopt it, according to the report. The companies might be willing to do this if they’re already using the standard, says Ms. Srikumar.The harder problem is figuring out how – and why – an agent made a particular decision. In other words, telemetry can show what an agent did, but it may not reveal why it made that choice. The gap exists because there are no reliable signals to determine that, according to the report. Such knowledge will be important in the future if, for example, a fraud regulator wants to determine the reasoning behind an agent’s recommendation to take no further action on suspicious activity. Deepen your worldviewwith Monitor Highlights.Politics with respectGet political stories with respectful analysis.There‘s a world of new ideas in everyBooks newsletter.There‘s more to life, enrich yours withCulture & Learning weekly.Follow humanity‘s discoveries withScience & Nature stories in your inbox.Gain a spiritual perspectivefrom the stories in your inbox.Want to understand the deeper impact of critical events? Learn the Monitor‘s insight.Already a subscriber? Log in to hide ads. Various polls suggest that public distrust of AI technology is building. The drumbeat of concern is largely focusing on cybersecurity, where agents break into other computer systems. If individual users begin to have problems with the new consumer-based AI agents, that could trigger another area of concern. But, so far, corporations’ growing use of them and the potential risk haven’t generated the same level of concern.“There is not going to be a lot of sunlight or visibility in how these agents are deployed” within companies, says Ms. Srikumar. “I don’t think there’s enough of a drumbeat.” ALREADY A SUBSCRIBER? LoginReal news can be honest, hopeful, credible, constructive.The Christian Science Monitor was founded in 1908 to lift the standard of journalism and uplift humanity. We aim to “speak the truth in love.” Our goal is not to tell you what to think, but to give you the essential knowledge and understanding to come to your own intelligent conclusions. Join us in this mission by subscribing.