Google TPU v6e (Trillium)

What it is

Google TPU v6e, also called Trillium, is a tensor processing unit (TPU) offered through Google Cloud. A TPU is an accelerator designed to run machine-learning computations, especially the matrix and vector operations that dominate neural-network training and inference. Google Cloud publishes a specification table for the v6e that gives per-chip compute, memory, and interconnect figures, which makes it possible to describe the chip at the level of individual units and at the level of a pod.^[1]

Origin and availability

Google Cloud said Trillium was announced earlier in 2024 and was generally available to Google Cloud customers as of that post.^[3] The same source states that Google used Trillium TPUs to train the new Gemini 2.0, and that a single Jupiter network fabric can carry over 100,000 Trillium chips.^[3] Those two statements place the chip in Google's own model-training work and in its data-centre network design: Gemini 2.0 is a model trained on the chip, and Jupiter is the fabric that connects large numbers of them.

Compute and memory

The specification table lists 918 teraflops of peak BF16 compute and 918 teraflops of peak FP8 compute per TPU v6e chip.^[1] BF16 and FP8 are floating-point formats with 16-bit and 8-bit storage; lower-precision formats reduce memory traffic and can let hardware perform more operations per second, while the choice of format depends on the model and the stage of training or serving. The table also lists 32 GiB of HBM capacity and 1,638 GB/s of HBM bandwidth per chip.^[1] HBM is high-bandwidth memory placed close to the accelerator, and capacity determines how much model state and activation data can stay on chip, while bandwidth determines how quickly that data can be moved.

Each v6e chip has one TensorCore and two SparseCores according to the table.^[1] A TensorCore is the unit that performs matrix multiply-accumulate operations; a SparseCore is a specialized unit for sparse computations. Google's architecture documentation says each v6e matrix multiply unit is 256 by 256 and supports 8-bit integer formats.^[2] That detail describes the shape and datatype support of the core matrix engine, which matters because matrix multiplication is the basic operation behind fully connected layers, attention, and many other neural-network components.

Interconnect and scale

Per chip, the table lists 800 GB/s of bidirectional inter-chip interconnect bandwidth and 100 Gbps of data-centre network bandwidth.^[1] The first figure describes how quickly chips inside a pod can exchange data with each other; the second describes the link from the chip to the wider data-centre network. The table lists 256 chips per TPU v6e pod.^[1] Google's documentation says a v6e slice can contain 1, 4, or 8 TPU chips per virtual machine.^[2] A pod is therefore the large physical grouping of chips, while a slice is the allocation unit a customer sees when attaching chips to a virtual machine.

Position in development

The v6e sits in Google's TPU line as a Google Cloud offering with published per-chip numbers and a pod design that connects 256 chips.^[1] Its configuration shows the two ends of accelerator design: per-chip compute and memory for local work, and inter-chip bandwidth plus data-centre networking for jobs that span many chips. Google's statement that Trillium trained Gemini 2.0 gives a concrete workload for the chip.^[3] The same statement about Jupiter carrying over 100,000 Trillium chips shows that the design can scale beyond a single pod.^[3] For users, the practical view is a virtual machine with 1, 4, or 8 chips, inside a pod of 256 chips, connected through a fabric that can reach much larger counts.^[2][1][3]