DeepMind's AlphaZero masters chess, shogi and Go

Background

Board games have long served as test beds for artificial intelligence, because their rules are fixed and their outcomes are unambiguous. Chess in particular is described in the preprint as the most widely studied domain in the history of AI ^[1]. Chess, shogi, and Go are not variations on a single problem: they use different boards, different pieces, and different rules, and a program designed around one of them does not carry over to the others by default. The usual pattern in the field had been to build a separate system for each game, so a method that could handle several games without being rebuilt was an open question.

What was published

On 5 December 2017, DeepMind posted a preprint to arXiv titled "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm" ^[1]. The preprint lists 13 authors, among them Demis Hassabis ^[1]. What it describes is a single algorithm rather than three separate systems, and that framing — one method, several games — is the central claim of the title.

How it worked

The algorithm was given only the rules of the game ^[1]. Everything else came from self-play: the system plays against itself, repeatedly, and uses the results of those games as the signal that drives its improvement ^[1]. In self-play there is no external opponent and no fixed set of positions to memorise; the system generates its own material and gets steadily harder to beat as it improves. Because the input was the rules and not a body of human play or hand-written advice about which moves are good, the same procedure could be pointed at chess, shogi, and Go without being redesigned for each of them ^[1]. The rules differ, but the learning loop is the same.

Results

Within 24 hours, the algorithm reached superhuman level in all three games ^[1]. In each of the three, it defeated a world-champion program ^[1]. That is the key comparison: the yardstick in every case was not an amateur or an average engine, but the best program then available for that game.

Why it mattered

The significance lay less in any single game than in what the result said about generality. A general method that has to be taught nothing but the rules, and that is not specialised for a particular board or piece set, nevertheless matched the strongest existing programs in three games — including one, chess, that the preprint calls the most widely studied domain in the history of AI ^[1]. A field that has spent decades on chess is a demanding place to test a method that knows nothing about chess in particular.

The work appeared as a preprint on arXiv, the form in which a great deal of machine-learning research circulates publicly before formal peer-reviewed publication ^[1]. Posting it made the description of the algorithm available for other researchers to read and discuss, and it put the claim — one algorithm, three games, superhuman play within a day — on the record in a form anyone could examine.