Skip to content
Neural Daily

ToolsNews

TPU 8tx and 8i Max: confidential inference, long context

🇭🇺 Olvasd el magyarul →

Google added two systems to its eighth-generation TPU lineup. TPU 8tx protects data and model weights in hardware even while in use, and TPU 8i Max puts two 8i chips in one package, with double the compute, SRAM and HBM bandwidth.

· 2 min read

A Google TPU 8tx és TPU 8i Max rendszerlapjai egymás mellett sötét háttéren
Image: Google · Press kit · Source

What Google announced

Amin Vahdat, Google's SVP for AI and infrastructure, introduced two new TPU systems on November 9. They join the TPU 8t (training) and TPU 8i (real-time reasoning) announced in April. It is the first time Google has introduced four TPU systems in a single year (Google).

DCD notes these are not new chips: Google reconfigured compute, networking, storage and packaging around the 8t and 8i, which is why it calls them systems (DCD).

TPU 8tx: confidential computing

Data is usually encrypted at rest and in transit, but sits unencrypted in memory while it is processed. TPU 8tx changes that. It uses the same silicon as TPU 8t with 5th Generation AMD EPYC hosts and builds protection into the hardware, so data and model weights stay isolated even in use.

It also adopts the low-latency Boardfly network from 8i, which Google says gives up to 2x lower latency than 8t with a near-zero performance penalty. The same system can run and customize models: fine-tuning on private data, distillation and RL training for agents, without moving data between clusters. The target is regulated enterprises and governments.

TPU 8i Max: against the memory wall

When dozens of agents work across a whole codebase or hours of video, memory bandwidth becomes the bottleneck. TPU 8i Max combines two TPU 8i chips in one package: double the compute, double the SRAM and double the HBM bandwidth of 8i. Twice the silicon fits in the same rack footprint, which means more compute per square foot and per watt.

Why it matters

Google says TPUs now have nearly 100 external customers, including frontier labs, trading firms and universities. For developers, PyTorch support is the entry point; we covered TorchTPU, which brings native PyTorch to TPUs (TorchTPU brings native PyTorch and Hugging Face to TPUs). TPU 8tx matters most where data protection has so far kept models on sensitive data out of the cloud. Google did not give pricing or availability dates in the announcement.

    By Mikola Máté Patrik

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles