Perceptrons: An Introduction to Computational Geometry

The problem the book took on

*Perceptrons: An Introduction to Computational Geometry* was written by Marvin Minsky and Seymour Papert and published in 1969^[1]. The MIT Press describes it as appearing ten years after the discovery of the perceptron, as an analysis of what perceptrons can and cannot compute on specific tasks^[2]. The perceptron was a simple learning machine loosely inspired by the brain, and claims about its powers were widespread; the book set out to replace those claims with proved statements.

The MIT Press calls the result the first systematic study of parallelism in computation, saying that Minsky and Papert provided a mathematical analysis showing the limitations of a class of computing machines that could be considered as models of the brain^[2].

How the argument works

The crux of the book is a series of mathematical proofs. Minsky and Papert acknowledge some strengths of perceptrons, then show that the devices have major limitations^[1]. To make proof possible, the book studies restricted perceptrons: a retina, a single layer of input functions, and a single output, with limits placed on their "order", meaning the maximum number of input connections a unit may draw on^[1]. In other words, instead of arguing about perceptrons in general, the authors bound what each unit is allowed to see, and then ask which predicates are still computable.

The results

The most important limitation concerned the computation of predicates such as the XOR function and the connectedness predicate^[1]. Minsky and Papert proved that a single-layer perceptron could not compute parity under conjunctive localness, in a result labelled Theorem 3.1.1, and that the order needed to compute connectivity grew with the size of the input^[1]. Wikipedia lists Theorem 3.1.1 as stating that the parity function has order |R|^[1].

The style of the argument is what made it durable: the claims are not empirical impressions about particular machines but statements about whole classes of devices, derived from what each unit is permitted to look at.

Why it mattered, and what it was said to have done

Wikipedia notes that the book is often thought to have caused a decline in neural net research in the 1970s and early 1980s, and marks that causal claim with the words "It is claimed"^[1]. In the foreword to a later printing, Leon Bottou writes that the authors' rigorous work and brilliant technique does not make the perceptron look very good, and that perhaps as a result research turned away from the perceptron^[2].

The authors themselves rejected the causal story. In the prologue and epilogue of the 1988 edition they argued that perceptron research waned in the 1970s not because of their book but because of inherent problems with credit assignment and knowledge representation^[1].

The 1988 expanded edition

After the revival of neural networks, an expanded edition appeared in 1988. It added a chapter countering criticism that had been made in the 1980s, but contained no new scientific results^[1]. The MIT Press records that Minsky and Papert added a new chapter in 1987, in which they discuss the state of parallel computers^[2].

The 1988 expanded edition was reissued in 2017 with a new foreword by Leon Bottou, published by The MIT Press on September 22, 2017, in 316 pages^[2].

What came after

Subsequent work answered the book on its own ground. Research on three-layered perceptrons showed how to implement functions like XOR, and it was widely known that networks of such elements can compute any Boolean function^[1]. PDP researcher David Rumelhart and colleagues claimed in a 1986 report to have overcome the problems presented by Minsky and Papert^[1].

The book's lasting role is therefore double: it fixed a set of negative results that later architectures had to get around, and — by making the limits of single-layer devices precise — it clarified what a network with more layers would have to do.