NVIDIA announces the Blackwell platform

What NVIDIA announced

On 18 March 2024, at GTC, NVIDIA announced the Blackwell platform, the successor to its Hopper architecture, which had arrived two years earlier ^[1]. The central claim was an economic one: NVIDIA said Blackwell would let organisations run real-time generative AI on language models with a trillion parameters at up to 25 times less cost and energy than Hopper ^[1]. The company also said Blackwell's technologies support AI training and real-time inference for models scaling up to 10 trillion parameters ^[1]. The architecture is named after the mathematician David Harold Blackwell ^[1].

The technologies

NVIDIA listed six technologies in the announcement ^[1]. The GPU itself holds 208 billion transistors and is built from two dies joined by a link running at 10 TB/s; putting a very large chip together from two pieces only works if the connection between them carries data fast enough for the pair to behave like one piece of silicon.

A second-generation Transformer Engine adds 4-bit floating point inference ^[1]. Transformer is the network design behind most current language models; storing weights and activations in fewer bits during inference cuts memory traffic and arithmetic, which is one route to cheaper serving, at the cost of some numerical precision.

Fifth-generation NVLink runs at 1.8 TB/s per GPU and can connect up to 576 GPUs ^[1]. NVLink is NVIDIA's dedicated link between GPUs, used when a model is too large for one device and has to be spread across many. A RAS engine covers reliability, availability and serviceability — the machinery for detecting and recovering from hardware faults, which matters more as more devices are tied together. The confidential computing feature, called Secure AI, protects data while it is being processed rather than only while it is stored or in transit. A decompression engine was announced alongside them ^[1].

The products

Four products were announced ^[1]. The B200 is the GPU. The GB200 Grace Blackwell Superchip pairs two B200 GPUs with a Grace CPU. The GB200 NVL72 is a liquid-cooled rack holding 72 Blackwell GPUs and 36 Grace CPUs, wired to act as one GPU with 1.4 exaflops of AI performance ^[1]. An exaflop is 10^18 floating-point operations per second; treating a whole rack as a single accelerator is aimed at models too large to fit on one board. The HGX B200 is an eight-GPU board ^[1].

Who planned to use it

NVIDIA named Amazon Web Services, Dell, Google, Meta, Microsoft, OpenAI, Oracle, Tesla and xAI among expected adopters, and published supporting comments from Sam Altman, Demis Hassabis, Mark Zuckerberg, Elon Musk and others ^[1]. AWS, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure were to be among the first clouds offering Blackwell instances, with products from partners starting later in 2024 ^[1].

Why it mattered

Blackwell succeeded Hopper, the architecture launched two years earlier ^[1]. Its pitch was not peak speed on a single training run but the cost and energy of serving a model of a given size, an expense that recurs with every request rather than being paid once during training. The scale figures point the same way: a 10-trillion-parameter ceiling, 1.8 TB/s between GPUs and NVLink domains of up to 576 GPUs describe how large a model can be and how far it can be split across devices ^[1]. The named clouds and hardware partners indicate that the platform was meant to reach customers through existing data-centre channels rather than only as a standalone component ^[1].