Google's new AI chips keep your data locked, even while they work on it

Google announced TPU 8tx and TPU 8i Max, two new systems in its eighth-generation family of TPUs. A TPU, or Tensor Processing Unit, is a chip Google designs just for AI work.

This is the first time Google has introduced four TPU systems in a single year. The other two, TPU 8t for training AI models and TPU 8i for fast reasoning, came out in April at Google Cloud Next.

• TPU 8t: built for training huge models.
• TPU 8i: built for quick, real-time reasoning.
• TPU 8tx: built for secure, confidential inference.
• TPU 8i Max: built for massive context windows.

Inference means running a trained AI model to get an answer. A context window is how much text, code or video a model can keep in mind at once.

Data is normally encrypted when it sits on a disk or travels over a network. But while a computer works on it, the data has to be unlocked in memory.

That unlocked moment is the gap TPU 8tx tries to close. Google says it builds security into the hardware layer, so sensitive data and model weights stay protected and isolated even while in active use. Model weights are the numbers that make up a trained AI model.

Who it is for: Google names regulated enterprises and governments that run frontier or proprietary models and AI agents next to their most sensitive data. Some of those customers, Google says, asked for specialized encryption for unique workloads.

In real life think of a bank that wants an AI to read private loan files. With a normal setup, the files are unlocked while the AI reads them. With TPU 8tx, Google says the hardware keeps them shielded during that step too.

TPU 8tx is built on the same silicon as TPU 8t. It is paired with 5th Gen AMD EPYC processors as the host. A host processor is the main CPU that prepares data and hands jobs to the AI chips.

AMD announced the same day that its EPYC chips power the host side of TPU 8tx. AMD calls it the first cloud TPU system with hardware-enforced confidential computing. Google's Mark Lohmeyer said customers get "silicon-level data protection alongside the low-latency performance needed for modern reasoning models."

AMD says its Secure Encrypted Virtualization (SEV) features extend protection beyond the CPU to the connected accelerators. The system also checks an accelerator's identity and security setup before it gets direct access to protected memory. Prompts, model weights and in-between data stay encrypted in CPU memory and while moving between host and accelerators.

The big picture: the two companies have worked together for years. AMD says EPYC already runs under Google Cloud's confidential virtual machines and its confidential Kubernetes service, so TPU 8tx extends that into AI chips.

Google says speed is part of the pitch. TPU 8tx uses the same low-latency network, called Boardfly, that Google built for real-time reasoning. Google says that gives up to 2x lower latency than TPU 8t, with a near-zero performance penalty. Latency, which means the wait between asking and getting a reply, is what the network is built to cut.

That matters for AI agents, which take many steps in a row. Each step waits for the one before it, so small delays add up.

TPU 8tx is also good for small-scale training, Google says. Teams can fine-tune a model, which means teaching it more using their own private data. They can shrink a big model into a smaller one. Or they can use reinforcement learning, which means training an agent by rewarding good results. All of it runs on one system. Nobody has to move sensitive data between separate clusters.

TPU 8i Max tackles a different problem Google calls the memory wall. When dozens of AI agents work across a whole codebase or hours of video, memory speed becomes the bottleneck.

Google's fix is simple to say. It took two low-latency TPU 8i chips and put them on a single package.

• Compute: double that of TPU 8i.
• SRAM: double. SRAM is the very fast memory sitting right on the chip.
• HBM bandwidth: double. HBM is high-bandwidth memory, the big memory stack beside the chip.

Because it fits twice the silicon in the existing rack footprint, data centers can run more computing power per square foot and per watt, Google says.

Google says it wants TPUs to be as easy to use as ordinary hardware. A few pieces came with the announcement:

• TorchTPU: open-sourced last month. PyTorch code and Hugging Face models now run natively on TPUs, with no rewrite.
• TPU agent suite (preview): open-source tools that plug into coding assistants such as Claude Code, Cursor and Google Antigravity. They help with TPU setup and speed problems using plain language.
• XLA Serving Engine: train in Python, then serve in a fast C++ runtime across TPUs, GPUs and CPUs.
• Agent Compute Engine: pauses idle agent sessions and moves their working memory off the chip in milliseconds, which cuts costs.

By the numbers: Google says its TPU Sync layer cuts KV transfer delay and time to first token by more than 90%. A KV cache is the short-term memory a model builds while it reads a long prompt. Time to first token is how long you wait for the first word of an answer.

Two more tools are now generally available on Google Kubernetes Engine, which is Google's service for running software in containers.

• Cloud Pathways: lets you run large TPU jobs across many slices as if they were one machine. It brings the runtime behind Gemini to every Google Cloud customer.
• GKE dynamic slicing: assembles or splits TPU groups on the fly using optical circuit switching. Google says job start times drop by up to 5x and recovery from faults speeds up by up to 4.5x.

Google says TPUs have a decade of history. It first designed them for internal needs like voice recognition and translation. Today they train and serve its Gemini models, run agent workloads, and power Google Cloud for customers.

Google also lets customers put TPUs in their own data centers. It says TPUs are a growing business with nearly 100 customers, including frontier labs, trading firms, retailers and universities.

Its answer to the pace of AI is modularity. A new chip takes years of work on silicon. A new system, which rearranges packaging, memory, networking and security around a chip, can arrive in months. That is how Google says it shipped four systems in one year.

Google adds that it uses AI to design its chips too. It cites AlphaChip and Gemini producing chip layouts in hours instead of weeks.

• You will not buy these. TPUs are rented through Google Cloud, and the announcement gives no launch date or price.
• Private data may get easier to use with AI. If a company holds sensitive records, hardware protection could be a reason to say yes.
• Developers get an easier path. PyTorch code runs on TPUs without a rewrite, so trying them costs less effort.
• Watch the rollout. Availability and pricing for TPU 8tx and TPU 8i Max are the next facts to look for.