NVIDIA GB300 NVL72

A rack that behaves as one computer

On 18 March 2025, at its GTC event, NVIDIA announced Blackwell Ultra and two systems built on it: the rack-scale GB300 NVL72 and the HGX B300 NVL16.^[1] The GB300 NVL72 is the larger of the two. It packs 72 Blackwell Ultra GPUs and 36 Arm Neoverse-based Grace CPUs into a single rack, wired so that the whole cabinet works as one very large GPU.^[1]

That is the idea behind the "NVL72" in the name. Rather than treating a server as the unit of design and joining servers with ordinary networking, the rack itself becomes the machine. Software sees one pool of compute and memory instead of 72 separate accelerators. The approach matters most for very large models, which usually have to be split across many chips; the closer the chips sit and the more memory they share, the cheaper that splitting becomes.

What the numbers say

NVIDIA's central claim is that GB300 NVL72 delivers 1.5 times more AI performance than the previous GB200 NVL72 rack.^[1] Measured against earlier Blackwell GPUs, the same rack offers 1.5 times more dense FP4 Tensor Core FLOPS and twice the attention performance.^[2] FP4 means weights and activations are stored in four-bit floating-point numbers: lower precision, but more of them processed per second by the Tensor Cores. "Dense" describes a count without sparsity tricks; the specification table separately lists 1,440 petaflops of FP4 Tensor Core performance with sparsity for the full rack.^[2] Sparsity means skipping large numbers of parameters that are zero and computing only the parts that carry information.

Memory is the other half of the story. The specification table lists 37 TB of fast memory for the rack and up to 576 TB/s of bandwidth.^[2] Bandwidth is often the real limit in both training and inference, because moving weights and caches between chips can cost more time than the arithmetic itself. Networking is sized to match: each GPU gets 800 Gb/s of connectivity through ConnectX-8 SuperNICs.^[2]

Efficiency per megawatt

NVIDIA says GB300 NVL72 delivers 10 times better user responsiveness and 5 times better throughput per megawatt than Hopper systems, which it presents as a 50 times overall increase in AI factory output.^[2] Throughput per megawatt has become a headline figure because electrical power, more than floor space, tends to be what limits a datacentre: how many tokens a facility can produce for a given power draw decides what it is worth to run.

Availability and software

NVIDIA said products based on Blackwell Ultra were expected to be available from partners starting in the second half of 2025.^[1] Alongside the hardware, the company announced Dynamo, an open-source inference framework.^[1] Serving a model to many users is a different problem from training it, and NVIDIA paired the two announcements deliberately.

Where it sits

The GB300 NVL72 continues the rack-scale format introduced with GB200 NVL72, with the gains coming from a newer GPU plus more memory and network bandwidth per rack.^[1][2] It sits at the top of the Blackwell Ultra line, while the HGX B300 NVL16 puts the same generation into a separate, smaller system.^[1] All of the figures above come from NVIDIA's own materials; how they translate into real workloads will be measured once shipping systems reach customers and independent benchmarks appear.