AlphaGo

What it is

AlphaGo is a computer program built to play Go, a board game that had long resisted machine play. It was developed at DeepMind, which revealed it alongside a paper in Nature dated 28 January 2016, titled "Mastering the game of Go with deep neural networks and tree search" and authored first by David Silver and Aja Huang. The public announcement was made on 27 January, timed to the paper's publication^[1].

How it worked

AlphaGo combined two kinds of neural network — a policy network and a value network — with Monte Carlo tree search, a method for exploring possible future moves by sampling. The policy network suggests where to play; the value network estimates who is ahead. The tree search uses both to look ahead without examining every possibility, which is infeasible in Go because of the number of legal positions.

The networks were not hand-programmed with Go knowledge. They were first trained on games played by humans, and then improved through self-play, in which the program plays against versions of itself^[1]. According to DeepMind, AlphaGo was first introduced to thousands of expert games, then made to play against different versions of itself thousands of times^[2].

Early results

Against other Go programs, the single-machine version of AlphaGo won all but one of 500 games, and the distributed version won all 500^[1]. In October 2015 it played the European champion Fan Hui and won 5–0^[1].

The Lee Sedol match

In March 2016 AlphaGo played Lee Sedol, one of the strongest Go players in the world, in Seoul, and won the five-game match 4–1. The Korea Baduk Association gave it an honorary 9-dan rank^[1]. DeepMind says the match was watched by over 200 million people worldwide^[2]. One move from game two drew particular attention: DeepMind says Move 37 had a 1 in 10,000 chance of being played^[2].

Later versions and retirement

An updated version, AlphaGo Master, played top professionals online starting 29 December 2016 and stood at 60 wins and no losses as of 5 January 2017^[1]. In May 2017, at the Future of Go Summit in Wuzhen, China, AlphaGo beat Ke Jie 3–0 and received a 9-dan rank from the Chinese Weiqi Association. DeepMind then retired it from competitive play^[1].

It was succeeded by AlphaGo Zero, described in Nature on 19 October 2017, which learned without human game data^[1].

Why it mattered

Go was long considered a hard target for machine play because the number of possible positions is enormous and because strong play relies on judgment that is difficult to state as rules. AlphaGo's approach — learning from examples and from self-play, then searching with the guidance of learned networks — showed that this kind of judgment could be approximated by neural networks. The matches against Fan Hui, Lee Sedol and Ke Jie, and the public attention they received, made AlphaGo a widely known demonstration of what learned systems could do.