New AI Can Figure Out What You’re Thinking About From a Brain Scan

Allen et al./Nature Neuroscience Better learn about “brain privacy,” because it may soon go the way of the dodo. Researchers have developed an AI tool that can predict what you’re looking at by analyzing your brain scans — and then generate a strikingly accurate recreation of what you’re picturing. It’s barely an exaggeration to say that it’s a “mind-reading” tool, though it currently doesn’t work as seamlessly as it might in a sci-fi novel. “The results seem very impressive,” Tommy Sprague, a neuroscientist at the University of California, Santa Barbara, told MIT Tech Review. “But if there’s a way to surreptitiously extract information about what you’re thinking about, then… 150 years of sci-fi can come true anytime, and that’s worrisome in a lot of ways.” It’s not the first research to recreate images in someone’s head from brain scans. But Michal Irani at the Weizmann Institute of Science, whose team developed the tool, says their work is able to create a higher fidelity representation of what’s being pictured. For the work, which was recently presented at two conferences, the researchers trained an AI model on a public data set of brain scans that were taken from eight subjects while they were shown thousands of images. Unlike with older scans, they chose newer ones with a high enough resolution to show brain volumes as small as one cubic millimeter of neurons. The resulting “brain decoder” comprised one branch for predicting the structure of an image and another to predict its content, per MIT Tech. Together, they let an AI image generator known as a diffusion model recreate what the person saw. That was just the start. To really refine an AI model, you pump it with more data, and there simply isn’t enough of the specific brain scans they needed to go around. To work around this, the researchers trained an AI model that worked in reverse: one that predicts brain activity from an image a person is looking at. This “encoder” model can be used to spit out tons of predicted brain scans based on each specific image — which can then be used by the decoder model to recreate the image based on the synthetic scan. Rinse and repeat until the image more closely and closely resembles the original, and it was no longer necessary to use only images that were shown during actual brain scans. It’s not perfect. “Of course we have failures,” Irani told the magazine, noting how the tool reconstructed a dog in a bathtub as a goat in a bathtub. On the one hand, this technology could be a gamechanger for paralyzed people who are unable to communicate physically. On the other hand, it could one day spell the end of brain privacy. Right now, the work requires high resolution brain scans taken with massive and expensive fMRI machines. But if the same techniques were applied to something like a wearable EEG device, it would be a “game changer,” Marcello Ienca, a neuroscientist and philosopher at the Technical University of Munich, Germany, told MIT Tech. These devices can be calibrated to your particular brain — making it easy, in theory, for a company to pull information from your brain without consent. “I have no doubt that this is, you know, well-intentioned research, but I think it’s also pretty obvious that it could be co-opted for… ethically and societally problematic commercial uses,” he told MIT Review. More on brains: You’ll Never Guess Which Political Ideology People With Low Cognitive Ability Are Drawn Toward, According to New Research