NVIDIA B200 (Blackwell)

What the B200 is

The B200 is NVIDIA's data-center GPU of the Blackwell architecture, announced on 18 March 2024 at the company's GTC event, and the successor to the Hopper generation of accelerators.^[1] It is the compute chip at the centre of NVIDIA's AI systems: the part that carries out the matrix arithmetic behind training and running large neural networks.

The architecture is named after David Harold Blackwell, a mathematician who worked in game theory and statistics and was the first Black scholar inducted into the US National Academy of Sciences.^[1]

One GPU built from two dies

Each Blackwell GPU packs 208 billion transistors and is manufactured on a custom TSMC 4NP process. It is not a single monolithic piece of silicon: each GPU is made of two dies at the reticle limit — the largest area a lithography tool can expose in one shot — joined by a chip-to-chip link running at 10 TB/s, so that the pair behaves as one unified GPU.^[1] Splitting a large chip into smaller dies is a way of getting past the size ceiling imposed by the manufacturing equipment, at the cost of needing a very fast connection between the halves.

Transformer Engine and FP4

A second-generation Transformer Engine adds 4-bit floating point (FP4) arithmetic for inference. NVIDIA said this doubles the compute and the model size the chip can support.^[1] Number formats matter because a model's weights and intermediate values have to be stored and moved through memory; using fewer bits per value leaves more room and more bandwidth for the arithmetic itself, which is often what limits inference.

NVLink and on-chip additions

Fifth-generation NVLink provides 1.8 TB/s of bidirectional throughput per GPU and can connect up to 576 GPUs.^[1] Fast links between accelerators matter because a large model is usually spread across many chips, which then have to exchange data constantly. The chip also adds a RAS engine for reliability and a dedicated decompression engine aimed at data analytics workloads.^[1]

Rack-scale systems

The B200 is sold less as a standalone card than as part of larger units. The GB200 Grace Blackwell Superchip connects two B200 GPUs to one NVIDIA Grace CPU over a 900 GB/s chip-to-chip link. The GB200 NVL72 rack combines 36 of these superchips — 72 Blackwell GPUs in total — and acts as a single GPU with 1.4 exaflops of AI performance and 30 TB of fast memory.^[1] NVIDIA claimed that the NVL72 delivers up to 30 times the large language model inference performance of the same number of H100 GPUs, and cuts cost and energy use by up to 25 times.^[1]

At the level of a single server, the DGX B200 holds eight Blackwell GPUs with 1,440 GB of HBM3e memory in total, 64 TB/s of memory bandwidth, 72 petaflops of FP8 performance in the sparse form (dense is half that) and 14.4 TB/s of aggregate NVLink bandwidth, drawing about 14.3 kW. NVIDIA says it delivers three times the training performance and 15 times the inference performance of a DGX H100.^[2]

Who expected to use it

Organisations named at launch as expected adopters included Amazon Web Services, Dell, Google, Meta, Microsoft, OpenAI, Oracle, Tesla and xAI. Mark Zuckerberg said Meta would use Blackwell to help train its Llama models.^[1]

Where it sits

The B200 continues a pattern in which each NVIDIA generation is sold as a whole system rather than a lone chip: the GPU, its interconnect, its memory and the rack it sits in are designed together. The figures above are performance claims made by NVIDIA at and around launch; what they mean in practice depends on the software, the model and the way the cluster is wired. Its place in the line is simpler to state: it is the successor to Hopper, aimed at the same job of training and serving very large models, with more transistors, a wider interconnect and a lower-precision arithmetic path for inference.^[1]