DeepMind describes MuZero, which plans without knowing the rules

Background

Game-playing has been a proving ground for AI because games give clear goals and measurable outcomes. A recurring question is how much of the game a system must be given in advance. MuZero was introduced as a system that plans with a learned model rather than known rules.^[1] That framing puts the emphasis on learning a model useful for planning, instead of relying on a hand-supplied rulebook.

What was posted

On 19 November 2019, DeepMind posted “Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model” to arXiv.^[1] The preprint lists 12 authors, including Demis Hassabis and David Silver.^[1] A revised version followed on 21 February 2020.^[1] These details place the arXiv paper as the public starting point for MuZero: a preliminary paper that described the approach.

Planning with a learned model

Planning means looking ahead before acting: considering possible sequences of events and using them to choose a move. Many game systems need explicit rules to generate those possibilities. MuZero's distinguishing point is that its planning uses a learned model, not rules supplied in advance.^[1] In other words, the model that supports the lookahead is learned rather than written down as a game rulebook. This matters because the same general approach can be applied to games whose rules are not handed to the system.

Results

The preprint reported a new state of the art on 57 Atari games.^[1] It also matched AlphaZero in Go, chess and shogi without being told the rules.^[1] The second result is the clearer comparison: AlphaZero is used as a reference point, and MuZero reached comparable performance in those three board games while relying on a learned model for planning.^[1] The Atari result shows the approach was not limited to board games: it was tested across a large set of video games.^[1] Together, the claims frame MuZero as a single system for planning across different kinds of games.

Where it fits

The public record has two stages. The 2019 arXiv preprint was revised in February 2020.^[1] DeepMind's MuZero post, dated 23 December 2020, describes MuZero in a Nature paper, after the 2019 preliminary paper.^[2] That sequence makes the November 2019 posting the first public description of MuZero in these sources.^[1][2] Its importance lies in the combination of planning and a learned model: the system plans, but does so without being given the rules of the games it plays.^[1] For readers following game-playing AI, MuZero is a marker of a shift in emphasis—from systems that depend on explicit rules toward systems that learn a model and plan with it.