NVIDIA announces Blackwell Ultra

Background

A large language model does most of its work before it is deployed: training absorbs the computation, and answering a question is comparatively cheap. Test-time scaling inference shifts part of that emphasis. Instead of spending more compute only during training, the system spends more compute while it is answering — working through intermediate steps, or weighing several candidate answers before settling on one — and that extra computation is used to raise accuracy. NVIDIA described the idea in these terms, as applying more compute during inference to improve accuracy ^[1].

The shift has consequences for the machines underneath. A model that produces long chains of intermediate work needs somewhere to keep that work while it is being generated, and a service that answers many such requests needs throughput as well as raw arithmetic. That is the gap the new platform was aimed at.

What NVIDIA announced

On 18 March 2025, at its GTC conference in San Jose, NVIDIA announced Blackwell Ultra ^[1]. The company positioned the platform for two jobs at once: training, and test-time scaling inference ^[1]. Announcing both together reflects the argument that the same generation of hardware should serve the stage where a model is built and the stage where it is used.

The systems and the numbers

The system described at launch was HGX B300 NVL16. According to NVIDIA, it offers 11 times faster inference on large language models, 7 times more compute and 4 times larger memory than the Hopper generation ^[1]. Those three figures measure different things — speed on the inference workload, aggregate compute, and memory capacity — and NVIDIA presented all three as the improvement over the previous generation of its data-centre platform. A rack-scale configuration, GB300 NVL72, would also be available on NVIDIA DGX Cloud ^[1].

How it would reach customers

NVIDIA named the server makers it expected to deliver Blackwell Ultra-based servers: Cisco, Dell Technologies, Hewlett Packard Enterprise, Lenovo and Supermicro ^[1]. It also named the cloud providers that would be among the first to offer Blackwell Ultra-powered instances: Amazon Web Services, Google Cloud, Microsoft Azure and Oracle Cloud Infrastructure ^[1]. The two lists correspond to two routes to market — buying a system, or renting capacity by the instance — and together they indicate that the platform was meant to be available both as hardware and as a cloud service from the start.

Why it matters

NVIDIA chief executive Jensen Huang said that reasoning and agentic AI demand orders of magnitude more computing performance ^[1]. The claim links a hardware generation to a change in workload. If models are to spend more computation on each answer rather than only during training, then the computing capacity standing behind each answer has to grow with it, and the balance inside a system — compute against memory — has to be re-struck. Blackwell Ultra was presented as NVIDIA's answer to that rebalancing, and the partner lists show the company lining up the channel through which such systems are usually bought.