NVIDIA Cosmos world foundation models

What it is

NVIDIA Cosmos is a platform of world foundation models for physical AI, meaning AI that operates in the real physical world, such as robots and self-driving vehicles. NVIDIA announced it at CES on 6 January 2025.^[1] A world foundation model is a generative model that produces video of how a physical scene looks and moves, so that machines can be trained in simulated scenes before facing the real world.

The models generate physics-based video from several kinds of input: text, images, video, and robot sensor or motion data. The purpose stated by NVIDIA is to train robots and autonomous vehicles on synthetic data, meaning data created by computer rather than collected from the world.^[1]

Origins

The platform was launched at CES on 6 January 2025 as a set of open-weight models released under the NVIDIA open model license.^[1][2] The technical report, by 77 authors led by Niket Agarwal, was posted on 7 January 2025.^[2] Wikipedia records that Cosmos was presented alongside Isaac GR00T N1, a humanoid robot model, at GTC 2025.^[3]

Main features

The platform includes both autoregressive and diffusion model types.^[1] An autoregressive model generates its output one piece at a time, each piece depending on those before it. A diffusion model starts from noise and refines it step by step into an image or video. Offering both lets developers choose the approach that suits a given task.

The platform has four main components: the world foundation models, the Cosmos Tokenizer, guardrails and a video-curation pipeline built on NeMo Curator. A tokenizer converts video into compact tokens that a model can process. NVIDIA said the Cosmos Tokenizer compresses 8 times more and runs 12 times faster than leading tokenizers.^[1] Guardrails are safety filters that check inputs and outputs. The curation pipeline sorts and labels raw video so it can be used for training.^[1]

NVIDIA also gave a speed figure for the curation pipeline. It said the pipeline can process, curate and label 20 million hours of video in 14 days on Blackwell, NVIDIA's GPU architecture, against more than three years on CPUs.^[1]

Where it fits

Cosmos was aimed at companies working on robots and vehicles. Early adopters named by NVIDIA included 1X, Agility, Figure AI, Uber, Waabi and XPENG, among others.^[1] Jensen Huang said: “We created Cosmos to democratize physical AI and put general robotics in reach of every developer.”^[1]

Cosmos was presented at GTC 2025 together with Isaac GR00T N1, which places it alongside NVIDIA's work on humanoid robots.^[3] The open-weight release means the models can be downloaded and adapted by others.