Some Studies in Machine Learning Using the Game of Checkers

The question

Some Studies in Machine Learning Using the Game of Checkers is by A. L. Samuel and appeared in the IBM Journal of Research and Development, volume 3, issue 3, pages 210 to 229; the journal record gives 31 July 1959 as the date of publication, with the DOI 10.1147/rd.33.0210 ^[1]. The question behind it can be put in one sentence: can a computer be programmed so that it gets better at a task through experience, rather than being told in advance exactly what to do?

Why checkers rather than chess

Samuel explains that checkers rather than chess was chosen because the simplicity of its rules permits greater emphasis to be placed on learning techniques ^[2]. The simpler the game, the less of the design effort has to go into describing the game itself, and the more of it can go into the learning machinery.

Two ways to build a learning machine

The paper distinguishes two approaches. One is a neural-net approach, in which learned behaviour would be induced into a randomly connected switching net by reward and punishment ^[2]. The other, which the author calls more efficient, is to build a network designed to learn only certain specific things ^[2]. The experiments reported in the paper were based on the second approach ^[2].

What was done and what happened

The abstract reports that two machine-learning procedures were investigated in some detail using the game of checkers ^[2]. Enough work had been done, Samuel writes, to verify that a computer can be programmed so that it will learn to play a better game of checkers than can be played by the person who wrote the program ^[2]. The program learns this in a remarkably short period: 8 or 10 hours of machine-playing time, given only the rules of the game, a sense of direction, and a redundant and incomplete list of parameters whose correct signs and relative weights are unknown ^[2].

That list of starting materials is worth pausing over. The program is not handed the right values for its parameters; it is given a list that is both redundant and incomplete, and does not even know which way round the signs should go. What it has instead is the rules, a sense of direction — some rough notion of which way progress lies — and time to play.

Earlier programs in the same line

A footnote in the paper records the sequence of programs behind the work. The first operating checker program for the IBM 701 was written in 1952, was recoded for the IBM 704 in 1954, and the first program with learning was completed in 1955 and demonstrated on television on 24 February 1956 ^[2].

The conclusion

Samuel concludes that it is now possible to devise learning schemes which will greatly outperform an average person, and that such schemes may eventually be economically feasible for real-life problems ^[2]. He adds that the principles of machine learning verified by these experiments are applicable to many other situations ^[2].

Why the paper matters

What makes the paper notable is less the checkers program than the kind of claim it makes and how it supports it. Samuel does not argue in the abstract that machines might learn; he sets out two procedures, runs them on a game, and reports a concrete result — a program that ends up playing better than its author, from a starting point in which the parameters and their weights are unknown ^[2]. The paper also frames a design choice that recurs whenever a learning system is built: between a general net shaped by reward and punishment, and a narrower network deliberately built to learn only certain specific things, which Samuel judged the more efficient of the two ^[2]. And it records the earlier programs on which the experiment rested, so that the result can be read as a step in a line of work rather than an isolated demonstration ^[2].