Google reveals the Tensor Processing Unit

What the TPU is

A Tensor Processing Unit is an application-specific integrated circuit, or ASIC — a chip designed from the start to do one kind of work well, rather than to be programmable for any task. Google built its version specifically for machine learning and tailored it to TensorFlow, the company's framework for building and training neural networks ^[1]. The work was kept quiet while it was under way; Google later described it as a "stealthy project" that had begun several years before the public heard about it ^[1].

The announcement

In May 2016, Google Fellow Norm Jouppi disclosed the chip in a blog post titled "Google supercharges machine learning tasks with TPU custom chip" ^[1]. Two points in that post stood out. The first was that the TPU was not a plan on paper: the chips had already been running inside Google's data centers for more than a year ^[1]. The second was the claim about efficiency. Google said the TPU delivered an order of magnitude better-optimised performance per watt for machine learning, a jump it compared to fast-forwarding seven years — three generations of Moore's Law ^[1]. Moore's Law is the long-standing observation that the number of transistors that fit on a chip roughly doubles every couple of years; a gain worth three of those steps would normally take about seven years of ordinary process improvement to arrive.

Trading precision for throughput

One design choice sits at the centre of the chip. The TPU tolerates reduced computational precision ^[1]. Most of the arithmetic in a neural network is multiplication and addition: an input value is multiplied by a weight and added into a running total. Representing numbers with fewer bits makes each of those operations cheaper in hardware — fewer transistors per operation — so more operations per second fit into the same piece of silicon ^[1]. The trade is exactness: individual results are slightly less accurate than they would be with wider numbers, in exchange for far more of them being produced.

What it was already running

Google named three workloads. TPUs powered RankBrain, the system used for search result relevance, and Street View ^[1]. They also powered AlphaGo in its matches against Go world champion Lee Sedol ^[1]. The examples matter because they are different in kind: search ranking, Street View imagery and game playing.

The full technical account

The detailed description came in April 2017, in a paper prepared for the ISCA 2017 conference. It reported 65,536 8-bit multiply-accumulate units and a peak of 92 trillion operations per second ^[2]. On Google's own inference workloads — the work of running an already-trained model, as opposed to training one — the chip showed 15–30x speed and 30–80x performance-per-watt advantages over contemporary GPUs and CPUs ^[2]. The scope of those numbers matters: the comparison was made on inference, and on Google's workloads, not on every kind of computation.

What followed

Google kept developing the line. By April 2025, when it introduced the seventh-generation chip, Ironwood, the company said TPUs had powered its most demanding AI training and serving workloads for more than a decade ^[3].