How AI chatbots and fake photos are playing with your mind

People’s trust in artificial intelligence is taking a dangerous new turn. Researchers are warning that AI-generated photos, videos, and text summaries are successfully distorting people’s own memories. What is even scarier is that even when we know a machine created the content, our brains still believe the fake information.The MIT Photo and Chatbot ExperimentsMIT scientist Pattie Maes recently shared some alarming findings with the Washington Post. In her team’s study, a group of participants was shown a series of ordinary photographs. After a break, some people were shown the exact same pictures, while others were shown versions that had been altered by AI.When their memories were tested later, the people who viewed the AI-edited photos were far more likely to misremember the original images. Worse, they were overly confident that their false memories were completely real.Another study by the MIT Media Lab found that simply talking to an AI chatbot can mess with your head. People who were questioned by a generative AI chatbot produced over three times as many false memories compared to those who didn't interact with the AI at all. Warning Labels Do Not WorkA separate study by Georgetown University looked at how AI text summaries affect us. They found that when people read a misleading AI summary of a video, their ability to remember a simple detail, like a traffic sign, dropped from 83.6% down to just 44.8%.You might think that a warning label would keep your memory safe. However, the researchers found that even when users were explicitly told the summary was AI-generated, it did not stop the fake memories from forming. Other studies have even shown that AI deepfakes can plant false memories about made-up political scandals, successfully changing how people plan to vote.Why "Humans in the Loop" Isn't Enough Human memory has always been fickle and easily influenced, a fact psychologists have known since the 19th century. Generative AI makes this much worse because it often makes confident mistakes, known as "hallucinations."As banks, hospitals, and militaries rush to use AI, they often promise to keep "humans in the loop." This means a human expert is supposed to check the AI’s work before any action is taken. However, research shows that humans are basically flipping a coin when it comes to correctly spotting AI-generated content. Because of our spotty memories and tendency to trust polite, confident AI systems, even human experts are highly vulnerable to having their judgment distorted by bad AI output.Real-World DangersWhile experts are struggling to find a fix for this psychological loophole, the tech industry is racing ahead to build completely autonomous AI systems that cut humans out of the process entirely.Blindly trusting AI without questioning its output has already led to catastrophic real-world outcomes. For example, an American military strike on an elementary school in Iran killed more than 150 people earlier this year, including at least 123 children. According to an internal Department of Defense investigation reported by Bloomberg, the deadly strike was partially caused by an overreliance on a Palantir-developed AI system, which used outdated data to recommend the school as a target.If AI continues to warp what we remember, our ability to double-check its facts will disappear, leaving us to trust machines that are often confidently wrong.- Ends