Google Opens the Gates of AI Slop Hell

Google Gemini has a new feature: Letting users generate AI slop without visible watermarks to identify it. The official Twitter account for the LLM announced on Friday (news dump, everyone!) that users will soon gain the ability to “decide if your image, video, and music creations made with Gemini will have visible watermarks.” Now it’s a toggle. A few months ago, per Android Authority, Google VP Josh Woodward polled Gemini users on their most-requested features and found removing the visible watermarks on images generated by Google’s Nano Banana, Gemini’s image-generation model, was in the top 10. It’s hardly a shocker that many AI image generation enthusiasts’ top priority is removing visible evidence of AI usage. So what, if any, safeguards against generative AI abuse remain in Gemini? Google points to two technologies: C2PA (content credential) metadata and SynthID, an invisible watermark that encodes itself into images, video, and audio. It’s not like it takes much effort to remove a visible watermark from an image. Just crop or obfuscate it. C2PA, being metadata, is similarly trivial to remove. What’s more, most social media services strip some or all metadata from uploaded content anyways, meaning that anyone looking to spread disinformation doesn’t necessarily even have to take additional steps. SynthID is more robust. It works by encoding itself into the pixels of an image, and is designed to be resilient enough that it registers even after an image has been cropped, resized, and reduced in quality until it looks like a 1990s AOL Mail attachment. Ars Technica tested it, finding that while SynthID becomes less detectable with iterations, the image is “uselessly blurry by the time that happens.” Still, Ars noted that SynthID (by Gemini’s own not designed to withstand adversarial attacks. At least two commercial services, Rephrasy and SynthID Bypass, already claim to beat the technology. There’s also open-source tools and how-to guides. One author of one Medium post on the subject admitted it took “weeks, 123k image pairs, 200 plain Gemini images and a very specific understanding of spread-spectrum encoding” just to make changes to images that sometimes fooled decoders into missing confidence thresholds. Any way you slice it, this decision is another little landmark on the road towards total erosion of everyone’s ability to recognize AI-generated images at face value. That’s probably good for people profiting from AI slop, and bad for everyone else. Woodward tweeted on Friday that the watermark-removal feature will soon be rolled out “except in countries where it’s required by law to keep them.” Ah. Google didn’t immediately respond to a request for comment on this story, but we’ll update if we hear back.