Octo

What it is

Octo is an open-source generalist robot policy: a single learned controller meant to drive many different robots, rather than one machine built for one job. Its authors trained it on 800,000 robot trajectories^[1]. A policy is the part of a robot system that decides which action to take next from what the robot senses. Calling a policy generalist means the same set of weights is intended to cover many tasks, robots and scenes instead of being written for a single one.

How it works

The authors describe Octo as a large transformer-based diffusion policy that supports flexible task specification and flexible observation and action spaces^[1]. Three ideas sit inside that sentence. A transformer is a neural network architecture built around attention, which lets the model weigh different parts of its input against one another instead of processing it strictly in order. A diffusion policy produces actions by starting from noise and removing it step by step, rather than predicting an action in a single shot. Flexible task specification means the model does not hard-code one fixed way of telling the robot what to do, and flexible observation and action spaces mean it can take in inputs and produce outputs of differing dimensions, which matters because robots differ in how many sensors and joints they have. Physical Intelligence described Octo as a 93-million-parameter model that uses diffusion outputs^[2]; parameter count is a rough measure of model size, and by that measure Octo is a comparatively small model.

Out of the box and fine-tuning

The authors said Octo can control a diverse range of robots out of the box and supports efficient fine-tuning to new robot configurations^[1]. Fine-tuning means starting from weights that are already trained and continuing training on a smaller amount of data for a particular setup, instead of training from scratch. They released pre-trained checkpoints, that is, saved model weights, together with the full training and fine-tuning pipelines^[1]. Someone adopting the model can therefore begin from an existing policy and adapt it, and can also retrace how it was built.

Results and limits

The work was presented at Robotics: Science and Systems 2024^[1]. A separate evaluation appears in Physical Intelligence's description of its own model π0, where Octo serves as a comparison point. On that team's harder dexterous tasks, including hard table bussing, shirt folding and taking toast out of a toaster, Octo scored zero; on its easiest table-bussing task Octo scored 0.043 while π0 scored 0.971^[2]. A score near zero means the task was essentially not completed. These figures come from another team's task set, which is built around fine manipulation, and not from the authors' own evaluation of Octo.

Where it sits

Octo's main contribution is the combination of a generalist policy with an open release: pre-trained checkpoints plus the full training and fine-tuning pipelines were made available^[1], so others can start from existing weights rather than train a generalist robot policy themselves. Its design choices, a transformer backbone with diffusion action outputs and support for varying observation and action spaces, place it in the line of work that treats one model as the starting point for many robots and tasks. In later write-ups of generalist robot policies, it appears as a baseline against which newer models are measured^[2].