π0 (pi-zero)

What π0 is

π0 (pronounced pi-zero) is a generalist robot policy that Physical Intelligence published on 31 October 2024, describing it as the company's first generalist robot policy^[1]. A policy is the part of a robot system that turns what the robot perceives, and what it has been told, into motor commands. The release came with a paper, "π0: A Vision-Language-Action Flow Model for General Robot Control", available as arXiv:2410.24164^[2].

How it works

Physical Intelligence said π0 started from a 3-billion-parameter vision-language model and was adapted for real-time dexterous robot control^[1]. Vision-language models are pretrained on large collections of images and paired text, which gives them broad knowledge of objects, scenes and instructions before they ever drive a robot. π0 keeps that knowledge and attaches an action output to it.

The company said π0 augments the pretrained vision-language model with continuous action outputs through flow matching, a variant of diffusion models^[1]. Diffusion-style methods learn to produce a sample by starting from noise and removing it step by step; applied to control, the model gradually refines a continuous stretch of motion rather than choosing one action from a fixed menu. Speed matters here: the company said dexterous manipulation required π0 to output motor commands at up to 50 times per second^[1] — about one command every 20 milliseconds, fast enough for fingers and wrists to keep correcting as an object shifts.

The paper said π0 can be prompted with language instructions, follow a high-level vision-language model policy, and acquire new skills through fine-tuning^[2]. Those three properties describe a common division of labour: a slower, higher-level model decides what should happen next, and π0 carries it out.

Training data and robots

Physical Intelligence said the full training mixture combined open-source data with a dexterous-task dataset it collected across 8 distinct robots^[1]. The company's blog lists the robots as UR5e, bimanual UR5e, Franka, bimanual Trossen, bimanual ARX, mobile Trossen and mobile Fibocom^[1]. The paper said the model was trained on data from single-arm robots, dual-arm robots and mobile manipulators^[2].

The training tasks, according to the company, included bussing dishes, packing items into envelopes, folding clothing, routing cables, assembling boxes, plugging in power plugs and packing food into to-go boxes^[1]. These are ordinary chores whose objects vary in shape and whose motions need care, which makes them a broader test than demonstrations of a single repeated task.

Results

The company compared π0 with OpenVLA (7B) and Octo (93M) and reported that π0 was the best-performing model on all five evaluation tasks^[1]. It also reported a smaller comparison: π0-small, a 470-million-parameter model without vision-language pretraining, scored less than half of the full model, which the company presented as evidence for the value of VLM pretraining^[1].

Where it sits

Much work in robot learning trains a policy for one task at a time, on demonstrations of that task. A generalist policy is meant to cover many tasks, and several kinds of robot, with a single set of weights, and π0 is presented as exactly that: one model trained across single-arm, dual-arm and mobile platforms and then steered by language^[1][2]. Its recipe — a vision-language model as the base, high-rate continuous action output on top, and fine-tuning when a new skill is needed — is what the paper's title calls a vision-language-action model^[2].