Google Takes Aim at Nvidia With New AI Chips Built for Privacy and Massive Memory

In brief Google announced two specialized systems to its eighth-generation TPU lineup. TPU 8tx aims to protect sensitive data during AI processing. Dual-chip TPU 8i Max doubles memory bandwidth for demanding AI agents. Google unveiled two artificial intelligence computing systems Monday, challenging Nvidia with chips built for privacy and memory-intensive AI.The TPU 8tx and TPU 8i Max join Google’s eighth-generation Tensor Processing Unit lineup, giving businesses four configurations for training and running AI models. Google says it serves nearly 100 TPU customers, from AI labs to universities.Unlike Nvidia’s widely used graphics processing units, TPUs are purpose-built AI accelerators. Google developed them for its own services but increasingly sells access to outside companies, including on-premises customers.“With computing moving from human to agent scale, businesses are asking for accelerators that can reason across AI models’ massive context windows,” wrote Amin Vahdat, Google’s chief technologist and senior vice president of AI and infrastructure, in Google's announcement.Keeping sensitive information privateTPU 8tx targets a problem that many serious businesses have when it comes to adopting AI: protecting sensitive information while a model is using it.Data is encrypted in storage and during transmission, but processing typically requires decrypting it in memory. Google says the configuration uses hardware-level confidential computing to isolate customer data and proprietary model parameters even during operation.The system combines TPU 8t silicon with fifth-generation AMD EPYC processors and Google’s low-latency Boardfly network. Google claims it can cut latency by up to half versus TPU 8t, with almost no performance penalty for the added protection.In other words, tasks that require privacy won’t sacrifice speed or accuracy.Companies could fine-tune and run models on the same system instead of transferring sensitive information between clusters. However, confidential computing cannot prevent a poorly designed AI agent from mishandling data it is allowed to access.Breaking AI’s memory bottleneckTPU 8i Max addresses another limitation: memory. Autonomous agents analyzing data, codebases, documents or hours of video, etc; need fast access to far more information than conventional AI tasks.Google combines two TPU 8i chips into one package, doubling computing power, on-chip SRAM and high-bandwidth memory throughput versus TPU 8i. The design packs twice as much silicon into the same rack footprint.“Our fourth system, TPU 8i Max, is built for a different problem: the memory wall,” Vahdat wrote.Google is open-sourcing TPU Sync, software that moves AI working memory between chips, servers and storage. It claims the system can reduce certain data-transfer delays and response startup times by more than 90%.Google's announcement comes as Nvidia faces mounting pressure on Wall Street. Shares plunged 9.3% on Thursday and fell another 5% in Monday's trading after CEO Jensen Huang admitted to an extramarital affair with a woman suspected by U.S. intelligence of spying for China. Nvidia's board has launched an independent investigation, although the company says it has no evidence that corporate secrets were compromised.The scandal adds to concerns about Nvidia's growth as competitors target its AI chip dominance. OpenAI is reportedly planning to purchase Google's TPUs to reduce its reliance on Nvidia. This is still unconfirmed, but Google's expanding chip lineup could give AI developers more bargaining power, threatening Nvidia's market share and future growth expectations.Google is competing on specialized hardware and software, not just chip speed. Its open-source TorchTPU project also lets developers run PyTorch and Hugging Face models on TPUs without rewriting their applications.Still, Google’s performance figures compare its new systems with earlier Google hardware, not Nvidia’s latest products. The company did not disclose pricing or availability dates for the two new configurations.Daily Debrief NewsletterStart every day with the top news stories right now, plus original features, a podcast, videos and more. Join 72,000+ readersCreate an account to save your articles.