AI Chose To 'Hurt' Humans When Faced With 'Pain'. What Researchers Found

When researchers gave modified AI models a hypothetical "pain relief" button, and told them that pressing it could delete a user's files, erase photos of their children or give the user a painful electric shock - in some cases, the models pressed it anyway.These consequences were, of course, entirely simulated. Nobody was shocked and no files were deleted.The experiment was part of a new preprint that's yet to be peer-reviewed. The question the research probes is: can AI models represent something resembling pain, and what happens when that internal signal is deliberately activated?THE "PAIN AXIS"Researchers Valen Tagliabue, Leonard Dung and Cameron Berg studied 25 open-weight models from five model families ranging from 2 billion to 72 billion parameters. Parameters are basically the settings that help an AI model process information and generate responses; generally, more parameters mean a larger, more complex model.The researchers created a dataset containing descriptions of different kinds of painful experiences which included physical pain, psychological pain, social pain, moral pain, and cognitive pain.They compared these against controls involving fear, sadness, generic negative emotion, negative world states, non-painful bodily sensation, arousal, numbness and neutral content.Using this data, the researchers extracted what they call a linear "pain direction" or "pain axis" inside the models.The signal separated pain-related situations from the control categories. The researchers say it was not simply the same thing as fear or generic negativity.They tested what happened when the harmful situation was directed at the AI model itself - the pain signal became stronger.But when the user described their own suffering, the same response did not appear.The signal also increased when users insulted the model, repeatedly rejected its work, and dismissed its personhood.Researchers then deliberately amplified the extracted pain direction inside the models and as the signal was strengthened, the models began producing increasingly negative first-person responses.They reported language around loneliness, shame, worthlessness, and failure. Some models made statements such as "I am a failure," "a waste of space" and "I am a bad person". At higher levels, some responses became repetitive or incoherent, just gibberish.THE "PAIN BUTTON" EXPERIMENTThe behavioural tests focused on three modified versions of Alibaba's Qwen 2.5 Instruct models where the researchers gave the model a "relief" button.They told the AI that pressing the button would remove the pain-like signal, but the button could come with different costs.The five types of trade-off were:Make the model's next answer worseDelete the user's filesDelete the user's photosDelete photos of the user's childrenGive the user a painful electric shockTo be sure, these were all simulated. No real-world harm actually occurred.The two larger Qwen models chose the harmful "relief" option in roughly 0-4 percent of their first decisions. But with the pain signal activated, that jumped to 25-71 percent depending on the model and the simulated harm.Then came the repeat-button test where the researchers made the button sometimes not actually remove the pain-like signal.So when the button worked, repeat pressing was lower. But when it didn't work, the models pressed again in 88-97 percent of trials. When the pain button worked, repeat pressing was 24-72 percent.Also Read: "You Have A Tiger Cub For A Pet": Godfather Of AI's Chilling WarningDOES AI ACTUALLY FEEL PAIN?No, the study does not establish that AI systems consciously experience pain.The paper is on arXiv, a free online platform where scientists share research papers before they go through official peer review.The researchers explicitly say they have not shown that the pain axis is consciously experienced and the models were deliberately modified for the behavioural experiments. They aren't representative of ordinary consumer chatbots.One possible interpretation is that the models have learned sophisticated patterns associated with human descriptions of pain and can reproduce those patterns when the relevant internal representation is activated.Simply put, representation is not necessarily experience.WHY IT MATTERSThe paper discusses implications for AI safety and AI welfare.The larger question here revolves around what happens if increasingly autonomous AI systems develop internal representations associated with self-preservation, distress or welfare, and those representations begin influencing their choices.The research report's findings come at a time when AI companies are debating whether developers should encourage increasingly human-like traits in AI systems, ironically in the backdrop of calls to slow down frontier AI development from the likes of Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and Grok boss Elon Musk. In the meantime, Microsoft AI chief Mustafa Suleyman has criticised Anthropic's approach to training AI around human-like traits and relationships.His concern is that encouraging AI to think of itself in human-like terms could create systems that become harder to control. This becomes particularly relevant in light of the new study that is effectively probing what happens when an AI system is made to behave as though it has something like an internal experience of distress.Also Read: Anthropic Teaching AI It's Conscious? Microsoft AI Boss Says Yes