Cerebras Wafer Scale Engine

A chip that is the whole wafer

The Cerebras Wafer Scale Engine (WSE) is an AI processor whose defining decision is one of scale: rather than cutting a finished silicon wafer into many separate chips, it uses the entire wafer as a single chip ^[1]. Cerebras Systems unveiled it on 19 August 2019, and co-founder Sean Lie presented it at the Hot Chips conference ^[1].

Semiconductor manufacturing ordinarily ends with the wafer being diced into individual dies, because a speck of dust or a crystal defect can spoil a chip, and small chips waste less silicon when something goes wrong. The WSE takes the opposite route: the wafer is the chip. Much of the engineering behind it exists to make that route work.

Scale

The chip carries more than 1.2 trillion transistors on 46,225 mm² of silicon. Cerebras contrasted this with the largest GPU, which it put at 815 mm² and 21.1 billion transistors — a factor of 56.7 in area ^[1].

Cores, memory and network

Inside are 400,000 compute cores called Sparse Linear Algebra Cores (SLAC), designed around the kind of mathematics neural networks perform. Cerebras says the cores skip multiplications by zero, which are common in deep learning ^[1].

All memory sits on the chip. There is 18 GB of SRAM, distributed next to the cores and serving as the only level of the memory hierarchy, with 9 petabytes per second of aggregate bandwidth. Cerebras claimed this amounts to 3,000 times more on-chip memory and 10,000 times more memory bandwidth than the leading GPU ^[1].

The cores are connected by an on-chip two-dimensional mesh network called Swarm, with 100 petabits per second of aggregate bandwidth, so communication between cores never leaves the silicon ^[1].

Making a whole wafer work

TSMC manufactured the WSE on its 16 nm process ^[1]. CEO Andrew Feldman said the design had to solve long-standing problems that had limited chip size, among them cross-reticle connectivity, yield, power delivery and packaging ^[1]. These are the constraints that ordinarily keep dies small: a lithography tool exposes only a limited field at a time, so anything larger must be stitched together from several exposures, and every extra square millimetre raises the chance that a defect lands on the die.

Generations

Cerebras launched the WSE together with its CS-1 system in 2019, and followed it with WSE-2 in 2021 ^[2]. The third generation, WSE-3, announced in March 2024, has 4 trillion transistors, 900,000 AI cores, 44 GB of on-chip SRAM and 125 petaflops of peak AI performance, manufactured by TSMC on a 5 nm process ^[2].

Why it matters

The WSE is best read as an argument about where the cost of AI computation comes from. In a conventional system, work is spread across many chips, and moving data between them — through wiring, packages and circuit boards — takes time and energy. By keeping 400,000 cores and all the memory on one piece of silicon, linked by Swarm, Cerebras keeps that traffic on-chip ^[1]. The comparison figures the company gives, such as 56.7 times the area of the largest GPU and thousands of times more on-chip memory and bandwidth, are the company's own claims rather than independent measurements ^[1], and the difficulties of the design are the ones Feldman names: yield, power delivery and packaging ^[1]. Because a wafer-sized chip cannot simply be swapped for a smaller one, the practical question is whether the gain in on-chip communication outweighs those costs. The line has continued to grow since — WSE-2 in 2021, WSE-3 in 2024 — which shows the approach was carried through several generations rather than left as a one-off demonstration ^[2].