NVIDIA releases Nemotron 3 Ultra, a 550B open model

Background

The Super model, released on 10 March 2026, has 120 billion total parameters and 12 billion active.^[2] Ultra is the largest model in the Nemotron 3 family.^[1]

What happened

On 4 June 2026 NVIDIA released Nemotron 3 Ultra. It has 550 billion total parameters, of which 55 billion are active for each token. The design is a hybrid Mamba-attention mixture-of-experts model using LatentMoE, a variant of the mixture-of-experts approach that NVIDIA describes in its research materials.^[1] In a mixture-of-experts design only some of the expert sub-networks run for each token. The 55 billion active parameters therefore set how much computation each token receives, even though the full model holds 550 billion parameters.

Key details

The model supports up to one million tokens of context, which is the amount of text it can consider in one pass. It was pretrained in NVFP4, a 4-bit number format that reduces memory and compute needs during training.^[1]

NVIDIA claimed that Ultra reached 5.9 times the throughput of GLM-5.1 and 4.8 times that of Kimi-K2.6, measured at 8K input and 64K output tokens. Throughput is the amount of output a system produces in a given time, so these figures compare how fast each model generates text under that setting. The numbers are NVIDIA's own measurements.^[1]

Because the model is so large, the amount of work each token needs is the practical measure of its cost, and NVIDIA framed its speed claims around that measure. The release also made the model's materials public. Checkpoints, datasets and recipes were published openly on Hugging Face and GitHub, so developers can download the weights, study the training data and follow the training process.^[1]

Impact

The release brought a 550-billion-parameter model into the open-model line, with the training recipe and data published along with the weights.^[1]