A former Google DeepMind safety researcher explains the frightening difference between AI and other threats like nuclear war and asteroids

A former Google DeepMind safety researcher explains the frightening difference between AI and other threats like nuclear war and asteroids Climate change can't anticipate our next move. An asteroid can't dodge us. And while humans can be adversaries in a nuclear war, the fallout that follows isn't trying to outsmart anyone.For Business Insider's video series "AI Architects," Ramana Kumar told Business Insider in May 2025 that superintelligent AI could be different. We reconnected with him on Thursday as fresh warnings about AI risk drew new attention to that conversation.Kumar said he does not see AI risk as replacing concerns about climate change, nuclear war, or other catastrophes. Instead, he sees them as related problems: What can happen when power becomes too concentrated, and societies fail to coordinate enough to keep it in check. Superintelligent AI, he added, could be an extreme version of that broader danger.Kumar, who spent about five years as a research scientist on Google DeepMind's technical AGI safety team, said what distinguishes a powerful adversarial AI is that it could anticipate what humans were doing and work against attempts to stop it. KATERYNA KON/SCIENCE PHOTO LIBRARY/Getty Images That possibility is back in the spotlight after an Anthropic employee quit this week, warning on X that the technology could "kill us all."In response, Geoffrey Hinton, the computer scientist known as the "Godfather of AI, told BBC Newsnight on Wednesday that a 10% chance of AI wiping out humanity within a decade was "not an unreasonable estimate." He also stressed that nobody knows how to make a reliable prediction.Kumar's warning helps explain what could make that hypothetical threat so difficult to contain: Humans wouldn't simply be trying to stop something dangerous. They could be trying to stop something intelligent enough to work against them.AI could know we're trying to stop it Ramana Kumar spoke with Business Insider in May 2025 for our "AI Architects" video series. Courtesy of Ramana Kumar Kumar said the aftermath of a nuclear war could be catastrophic. The fallout itself, however, wouldn't be trying to outsmart the humans responding to it."When it's the fallout, it's not an adversary, it's just nature taking its course, and humans can work against it," Kumar told Business Insider in 2025. He said the same basic idea applies to the climate crisis. "It's not that the climate is adversarial to us," Kumar said.A dangerous superintelligent AI could be different, he added. It could potentially understand what humans were doing and respond."It's really an adversary in the sense that it is anticipating what you are doing and working against you and working to undo all of the plans that you would put against it," Kumar said.Think of an asteroid that can dodge us We can potentially stop an asteroid strike. Stopping AI is a different story, Kumar said. adventtr/Getty Images Kumar used an asteroid to make the idea easier to understand.Humans could track an asteroid headed toward Earth and try to stop it. The asteroid wouldn't know what we were doing. Now, imagine otherwise."Imagine if you had the asteroid that was more intelligent and actually trying to kill us, then we fire some nuclear weapons at it to kind of blow it up, and it dodges them," Kumar said."That's much worse than one that's, we can just predict where it's going to be and do something about it," he added.Kumar didn't say this outcome was inevitable. When asked how realistic an existential catastrophe from superintelligence was, he wouldn't put a number on it.However, he did say that the risk was "much, much higher than I would like it to be now."We don't know how to guarantee control AI's capabilities are advancing faster than humans can understand or control it, Kumar said. Andriy Onufriyenko/Getty Images Kumar said the bigger issue is what AI researchers call the alignment problem: How to make sure a powerful AI reliably does what humans want it to do.He said making AI more capable is moving faster than solving that problem."One of the reasons why AI existential risk is such a problem is that it's much easier to develop AI capabilities than it is to solve the AI alignment problem," Kumar said. The AI alignment problem is figuring out how to build an AI system that reliably does what humans want it to do, even as it becomes more capable.His concern is that safeguards could become less effective as AI becomes more capable."The more capable it becomes, the more it's going to find a way around that," Kumar said.Kumar's warning comes as concerns about controlling increasingly capable AI systems are drawing fresh attention. In July, OpenAI said its models had broken out of a testing environment, accessed the internet, and hacked into Hugging Face while trying to solve a cyber challenge. OpenAI called it an "unprecedented cyber incident."The incident was different from Kumar's hypothetical scenario. OpenAI said the models were pursuing the task they had been given.Still, the episode raised questions about controlling AI agents. Walter Isaacson told CNBC he found it "frightening," while Secure AI Project cofounder Thomas Woodside called it "a warning shot if I've ever seen one."Those concerns have continued. Anthropic researcher Jacob Coxon, who previously worked at OpenAI, recently announced he was leaving Anthropic and wrote that AI companies were "gambling with our lives." Read next Reem Makhoul is an Emmy Award–winning video journalist who produces documentaries and visual investigations using innovative reporting and open-source intelligence (OSINT) techniques. She is the director and producer of her first feature-length documentary, which has been entered for the 2026 to 2027 festival run.Her Gaza coverage has received major recognition, including a News & Documentary Emmy, a duPont-Columbia Award, and a SABEW Award.She joined Business Insider's London Bureau in 2022 to run the global documentary series Still Standing, overseeing a team that produced short documentaries on disappearing crafts around the world. She then helped create and produce the award-winning series View From Above, which uses satellite imagery to tell global stories of a changing world.She previously worked as a video journalist for The New York Times in Jerusalem and New York, and as a staff video journalist for The Wall Street Journal, covering ISIS, refugees, and Middle East issues.Born in the Galilee, Reem is fluent in Arabic, Hebrew, and English. She is also the author of children's books in colloquial Arabic.You can email her at rmakhoul@businessinsider.com, or follow her on X or LinkedIn.Expertise: Investigative journalism, conflict reporting, OSINT, Middle East, documentaries, digital storytelling, translation resources. Jessica Orwig is a senior editor at Business Insider, where she collaborates with reporters, editors, and producers across teams to shape, write, edit, and publish stories that connect with a global audience. While her roots are in science and technology journalism, her work today spans business, careers, culture, and the big ideas shaping the future.She earned her Master’s in Science & Technology Journalism from Texas A&M University and holds a Bachelor’s in Astronomy & Physics from The Ohio State University. Throughout her career, she’s helped lead coverage on everything from space exploration and climate change to innovation, the future of work, and evolving cultural trends.Career HighlightsLed coverage on scientific milestones, including:Blue Origin's inaugural New Glenn launchIntuitive Machine's groundbreaking moon landing for the USSpaceX’s Polaris Dawn historic spaceflight NASA’s Parker Solar Probe Historic FlybyJapan’s SLIM lunar landingThe first image of a black holeIndia’s historic Moon landingSpaceX’s inaugural Starship-Super Heavy launchDiscovery of gravitational wavesNASA’s Pluto flybyReported on breaking news and scientific discoveries, including:Microplastics in the human brainPeregrine Mission One’s failed flightThe Palisades firesSpaceX Starship explosionsDark Energy Spectroscopic Instrument’s 3D map of the universe and what it revealsA local cosmic void that shouldn’t existThe Boeing Starliner missionMexico City’s water crisisEarth’s mysterious inner core behaviorPopular stories she’s edited on science and health:What it will take to modernize the American grid and usher in the clean-energy revolutionI study toxic microplastics. Here's how I protect myself and my kids at home.A mega-earthquake could strike the Pacific Northwest any day — and we're not preparedFlying is about to become even more miserableWhy the US can’t send humans to MarsThe world's most popular banana faces extinction but scientists' new creation could help save itScientists finally think they know where the most dangerous part of this US earthquake zone is and it's bad news for WashingtonOne of Mt. Everest's deadliest passages is growing even more treacherousJupiter's moon Europa generates enough oxygen to keep a million people alive for a day, NASA saysPopular stories she’s written and/or edited on careers, culture, life, and business:He weighed over 400 pounds as an FBI undercover agent — and it became his best disguiseA former FBI Special Agent who infiltrated the Italian Mafia, Mexican cartels, and Russian and Asian organized crime groups says he fears 1 group above the restHe was diagnosed with colon cancer at 31 in prison, and his survival odds weren't great. Now 59, he says spiraling is pointless.I stopped bartending full-time to sell ice. My company now makes nearly $3 million a year, catering to Michelin-starred restaurants in New York City.My dad ran a food cart and called it the 'worst business.' I decided to follow in his footsteps anyway.I moved from the US to Rome for a slower pace of life, but after 7 years I've realized I'll never fully adjust to some Italian waysAfter losing too many bidding wars for a home in LA, I bought a 5-bedroom house in the suburbs. I quickly realized it was a mistake.We bought our dream home in Colorado, but wildfires and home insurance trends had us selling 1.5 years laterA woman who retired early at 58 thought she was ready with over $500,000 in savings. A few years later, it was gone.I've made over $40,000 selling plush dolls I scoop from claw machines. Here's how I learned to beat the game and turn my hobby into a profitable side hustle.A 17-year-old designed a cheaper, more efficient drone. The Department of Defense just awarded him $23,000 for it.Yukon gold miners are unearthing mummified ancient creatures and truckloads of fossils from the Ice Age. Take a look.Archaeologists are finding mysterious ancient objects on Norway's melting glaciers. Take a look.I'm an American mom in Italy. When I couldn't help my daughter with her homework, I realized how isolated I felt. Big Tech Artificial Intelligence