AlphaGo Zero

What it is

AlphaGo Zero is a Go-playing program developed by DeepMind. The company announced it on 18 October 2017, and the article describing it appeared in Nature on 19 October 2017^[1][2]. What separates it from earlier Go programs is where its knowledge came from: it learned to play by playing games against itself, starting from completely random play, and it never used records of human games^[1]. In its article, DeepMind called AlphaGo Zero arguably the strongest Go player in history^[1].

Where it came from

By the time AlphaGo Zero appeared, DeepMind had already built versions of AlphaGo that played at a high level, and two of them serve as yardsticks in the AlphaGo Zero story. One is the version that had defeated Lee Sedol; the other is AlphaGo Master, which had beaten top professionals in online games^[1]. Those earlier versions used two neural networks, while AlphaGo Zero used one^[1]. They were also much heavier on hardware. DeepMind's power-consumption chart listed AlphaGo Zero at 4 TPUs, against 48 TPUs for AlphaGo Lee and 176 GPUs for AlphaGo Fan^[1]; TPUs and GPUs are the chips on which neural networks are trained and run.

How it works

The program takes in very little. Its only input is the black and white stones on the Go board^[1]. A single neural network stands in for the two that earlier versions relied on^[1], and the program does not use rollouts^[1] — the technique of playing a position out to the end of the game, often with random moves, to estimate who is ahead. The network's own judgment of a position is used instead.

Learning happens through self-play. AlphaGo Zero plays millions of games against itself^[1]. Because both sides of every game are the program, it always has an opponent of exactly the right strength, and it always has a result to learn from: moves that led to wins become more likely, moves that led to losses less so. Nothing in this loop requires a human to label positions or explain strategy, which is why the starting point can be random play rather than a database of professional games.

Results

Progress was quick. After three days of self-play, AlphaGo Zero beat the version of AlphaGo that had defeated Lee Sedol by 100 games to 0^[1]. After 40 days it outperformed AlphaGo Master^[1]. DeepMind also reported that the program developed unconventional strategies and creative new moves^[1] — play that human players had not taught it, found by search over its own games.

Why it matters

The result mattered beyond Go because it removed a common assumption. Systems that reach expert level are usually trained on records of experts: human games, human text, human labels. AlphaGo Zero reached a level DeepMind described as arguably the best in history while using none of that, which suggests that in domains where human examples are scarce or expensive, the examples can in principle be generated by the system itself.

What came after

In December 2017, DeepMind generalised the approach to chess and shogi under the name AlphaZero^[2], taking the same self-play method off the Go board. AlphaGo Zero is therefore remembered less as a single strong Go engine than as a demonstration that self-play alone, given enough games and the right network, can produce play that people did not anticipate.