US calls for AI poisoning to sabotage China’s model distillation

Beijing has threatened to retaliate if Washington moves to curb Chinese artificial intelligence (AI) firms over accusations that they used a technique called distillation to copy AI models, days before senior officials from both countries meet for AI safety talks. Knowledge distillation is a technique in which a smaller “student” AI model is trained to mimic the outputs of a larger, more capable “teacher” model, allowing it to approximate the teacher’s abilities at a fraction of the cost. The dispute centers on a joint advisory from the US Federal Bureau of Investigation (FBI), the National Security Agency (NSA), and the Cybersecurity and Infrastructure Security Agency (CISA) accusing Chinese AI companies of using industrial-scale knowledge distillation to extract US intellectual property. The advisory named six Chinese firms, DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun and Z.AI, accusing them of extracting billions of tokens from Claude, ChatGPT, Google Gemini and Grok since late 2024 through fraudulent accounts and proxy services. The agencies said the campaigns were likely carried out with the Chinese government’s knowledge. “China-based AI companies are conducting systematic extraction of proprietary functionalities and capabilities of US AI companies’ models through industrial-scale knowledge distillation campaigns that form the core, not merely a supplement, of their AI development strategy,” the agencies said. The three agencies called on US AI firms to: Develop strategies to detect malicious prompts and suspicious accounts. Modify or restrict the responses given to accounts suspected of running distillation campaigns. Share information about malicious actors across the industry to close gaps exploited on multiple platforms. A less-noticed section of the advisory goes further than blocking or flagging accounts. “Employing targeted changes in response to high-confidence malicious distillation requests can impose meaningful costs on knowledge distillation campaigns. Response changes, such as including differential privacy or using less sophisticated ‘downgraded’ models to respond to distillation requests, can help protect US proprietary functionalities and capabilities and reduce payoffs from distillation attempts,” the advisory says. “Reducing reasoning depth, presenting correct information with different reasoning, or stylistic inconsistencies may evade detection while reducing training usefulness,” it says. “Avoid informing China-based AI company users suspected of distillation campaigns of a switch to a downgraded model. Informing malicious distillers would enable them to improve their defense evasions and indicate when to roll back training.” The advisory adds that firms should alter responses only for users confirmed to be querying frontier models for malicious distillation. In effect, the advisory’s message to US firms is simple: poison suspected Chinese “students” with bad data, or hallucinations in AI jargon, but never tell them they did so. “There is no day after tomorrow if China wins at this,” Bessent said at a Breitbart News economic forum in Washington on Tuesday. “If they were to pull ahead of us on AI, then nothing else matters.” He added that the US could not afford to pause AI development while China and other rivals pressed ahead. In fact, Washington started this poisoning tactic several months ago. In July, Bessent said the US had found watermarks from American models in Chinese systems and threatened sanctions. White House science adviser Michael Kratsios said China’s Moonshot AI had distilled Anthropic’s Fable model to help build its own K3 model using methods designed to evade detection. Neither Beijing nor much of the global media appears aware of the advisory’s quiet call for AI poisoning tactics. Chinese officials have instead responded forcefully to the prospect of US sanctions and to the broader distillation accusations. They said distillation is a neutral, widely used practice, and accused Washington of using national security as a pretext to protect its dominance in computing power and data. Foreign Ministry spokesperson Mao Ning said China’s AI progress stemmed from self-reliant innovation and open cooperation. “We hope the US side will earnestly act on the important common understandings reached between the two presidents and stop leveling false allegations to smear China,” she said. An unnamed Chinese Ministry of Commerce spokesperson called the US approach a double standard, noting that American firms had themselves drawn heavily on Chinese open-source models even as Washington accused Chinese firms of copying US systems. “This is a textbook case of using a crackdown on distillation as a pretext for industrial monopoly,” the spokesperson said. He said the US and Chinese state heads had agreed to launch an AI dialogue and that China was willing to engage on equal and mutually beneficial terms, but warned Beijing would take resolute countermeasures if Washington used distillation as a pretext to contain Chinese AI companies. The dispute is unfolding weeks before the two countries hold AI safety talks in Beijing, where Bessent will likely lead the American delegation and discussions are expected to focus on preventing AI-related security incidents. The talks come ahead of a broader summit between US President Donald Trump and Chinese President Xi Jinping planned for September 24 in Washington. A hard habit to break Despite the risk of their models being poisoned during distillation, Chinese AI developers are unlikely to break their habit of tapping US and Western AI models. Some Chinese commentators see distillation as the shortcut that lets the country’s AI companies catch up and break through the technological blockade formed by Washington’s export controls on advanced chips, in place since late 2022. “Model distillation itself is a technical method widely used across the global AI industry, and not all capability transfer or model learning can simply be equated with a national security threat,” Liu Dian, a research fellow at Fudan University’s China Institute, told the Global Times. He said that if Chinese firms could build similar-capability models more cheaply through open-source development and engineering optimization, Washington’s chip controls alone would not preserve its lead. He said even if the US sanctioned Chinese firms and banned their AI models, China should keep focusing on developing AI applications and a broader ecosystem. Bao Dequan, a Jiangsu-based columnist, says China takes pride in using distilled AI models at low cost, since its goal is not to nurture large AI model developers but to apply AI across industries to drive economic gains and industrial upgrading. “US AI runs on a closed capital loop, making money through monopoly premiums in hardware and LLMs, while Chinese AI is about industrial application, creating value by improving efficiency for the many,” he says. He says a US firm might charge a bank millions of dollars for one agentic system, while a Chinese firm might charge a garment factory only a few hundred thousand yuan for similar work. He says that gap explains why American AI needs high prices to survive while Chinese AI competes on value. Some observers say small Chinese AI developers would also like to charge users a monthly subscription, as OpenAI’s ChatGPT and Anthropic’s Claude do, but they have no choice amid fierce competition in China. The observers say these companies must let users run their LLMs for free because Chinese giants such as Tencent, Alibaba and ByteDance have embraced open source, backed by strong cash flow from cloud services. In the end, all LLMs end up being free and distillation is the cheapest way to build them, they say. Read: Nvidia chip export loophole clouds US-China AI summit talks Follow Jeff Pao on X at @jeffpao3