Cerebras Wafer-Scale Engine 3

What it is

The Cerebras Wafer-Scale Engine 3 — WSE-3 — is an artificial-intelligence chip that Cerebras announced on 13 March 2024^[1]. Ordinary processors are cut from a silicon wafer into many separate dies, each packaged and then wired to the others through boards and links. A wafer-scale engine takes the opposite route: it keeps one continuous piece of silicon. The WSE-3 covers 46,225 square millimetres^[2] and holds 4 trillion transistors and 900,000 AI-optimized compute cores on that single slab^[1].

What sits on the chip

Cerebras quotes 125 petaflops of peak AI performance for the WSE-3^[1]. A petaflop is a thousand trillion arithmetic operations per second, so the figure describes how much work the chip can do at peak. The chip also carries 44 GB of static random-access memory (SRAM) on the same piece of silicon and is manufactured on TSMC's 5-nanometer process^[1].

Keeping memory on the chip is central to the design. When a processor stores its working data in separate memory chips, every read and write travels over a link that is slower and more power-hungry than the arithmetic itself. AI workloads repeatedly move large arrays of numbers, so that traffic can dominate both time and energy. On-chip SRAM shortens the distance the data has to travel.

Memory beyond the chip

Forty-four gigabytes is modest next to the largest models, so the WSE-3 is paired with external memory in one of three configurations: 1.5 TB, 12 TB or 1.2 PB^[1]. Cerebras says the system can train models of up to 24 trillion parameters inside a single logical memory space, without partitioning^[1]. Partitioning means splitting a model across separate devices, which then have to exchange data constantly; avoiding that split is the main argument for building a chip this large in the first place.

Scaling out

Individual chips are combined into larger machines. Cerebras says clusters of as many as 2,048 CS-3 systems can reach 256 exaflops^[1], where one exaflop is a thousand petaflops. The arithmetic scales with the number of systems, which is what makes the design usable for training runs rather than only for single-chip work.

Against the previous generation and a competitor

Cerebras says the WSE-3 delivers twice the performance of the previous generation, the WSE-2, at the same power draw and price^[1]. The company also compares the chip with NVIDIA's B200, claiming 19 times more transistors and 28 times more compute^[2]. Both comparisons are Cerebras's own figures, not independent measurements, and they are best read as the company's positioning of its product.

Where it fits

The WSE-3 is one clear answer to a problem every AI chip designer now faces: as models grow, the cost of moving data between chips takes up a larger share of the total. The wafer-scale answer is to make the chip big enough that much of that traffic stays inside one piece of silicon, and to accept the manufacturing difficulty that comes with it. Most of the industry takes the other route, linking many smaller chips with high-speed connections. The two approaches are being pursued in parallel, and the WSE-3 is the third generation of the wafer-scale side of that split.