Google TPU v3

What the TPU v3 is

Google TPU v3 is the third generation of Google's Tensor Processing Unit, a chip the company designs itself to run machine-learning computation. Google introduced it in 2018 ^[1]. A TPU is an accelerator rather than a general-purpose processor: it sits in a datacenter alongside ordinary CPUs and takes on the dense numerical work — mainly large matrix and vector operations — that neural networks spend most of their time on. Google offers the chip to outside users through Google Cloud, so it serves both the company's own services and cloud customers ^[1].

What changed from the second generation

The clearest change is memory. The v3 doubles memory to 32 GB of HBM, high-bandwidth memory that stacks DRAM dies and connects them to the processor through very wide interfaces, and it raises memory bandwidth to 900 GB/s ^[1].

Peak throughput is listed as 123 trillion operations per second, up from 45 on the v2 ^[1]. Peak figures describe what the hardware can do under ideal conditions; real models also wait on memory traffic and on communication between chips, so the throughput reached in practice is normally lower. That is part of why memory capacity and bandwidth matter as much as raw arithmetic.

The v3 stays on a 16 nm manufacturing process ^[1]. Its thermal design power is 220 W, lower than the v2's 280 W ^[1]. Power draw sets how many chips a rack, and a datacenter hall, can hold; a lower figure per chip leaves room for more of them.

Liquid cooling

Google used liquid cooling on the v3, adopting it to handle the higher density of TPU v3 pods ^[1]. Air cooling moves heat from a chip's heatsink into the room, and it becomes harder as chips are packed closer together. A liquid system instead runs coolant through cold plates mounted on the chips and carries the heat away in the fluid. Because the coolant loop can be sealed and the heat exchanged elsewhere, machines can be packed more tightly than air cooling allows — which is what a denser pod requires.

Numbers and availability

Like the v2, the v3 supports bfloat16 ^[1]. bfloat16 is a 16-bit floating-point format that keeps the 8-bit exponent of a standard 32-bit float and shortens the mantissa, so the range of representable numbers stays wide while each value takes half the space. Models can therefore move activations and parameters with less memory traffic, at some cost in precision. Google offers the v3 through Google Cloud ^[1].

Where it sits

The v3 is the third generation of a chip line Google designs for machine-learning work and also offers through its cloud ^[1]. Compared with the v2, its gains came in memory capacity and bandwidth, in peak throughput, and in power per chip, while the manufacturing process stayed the same ^[1]. It also marks liquid cooling entering Google's TPU deployment, a change driven by packing more chips into a pod ^[1].