David Rumelhart

Overview

David E. Rumelhart was an American cognitive psychologist, a scientist who studies how people perceive, remember, learn and understand language. He was born on 12 June 1942 in South Dakota and died in 2011 in Chelsea, Michigan, aged 68.^[3] In the history of artificial intelligence his name is tied to one idea above all: a network of simple, neuron-like units can learn useful patterns from examples, instead of being programmed with explicit rules.

Education and early research

He earned a BA in psychology and mathematics from the University of South Dakota in 1963, and a PhD in mathematical psychology from Stanford University in 1967.^[3] Mathematical psychology describes mental processes with formal models, often equations or computer programs, that can be checked against experimental data. His later work drew on this background.

Main contributions

Much of Rumelhart's best-known work was collaborative. With James McClelland and the PDP (Parallel Distributed Processing) Research Group, he published the two-volume book Parallel Distributed Processing: Explorations in the Microstructure of Cognition in 1986.^[3] The book brought connectionist models, which explain mental abilities through the strengths of connections among many simple units rather than through symbols and rules, to a wide audience.

A key paper from the same period is "Learning representations by back-propagating errors", published in the journal Nature in October 1986. Rumelhart was its first author, with Geoffrey Hinton and Ronald Williams, and he wrote it while at the Institute for Cognitive Science at the University of California, San Diego.^[2]

Back-propagation works like this. The network turns an input into an output, and the gap between that output and the correct answer is measured as an error. The error is then passed backward through the layers, from output toward input, and each connection is nudged so that the next error is a little smaller. Suppose a network guesses the next letter of a word and guesses "a" when the answer is "e". Connections that pushed toward "a" are weakened a little, and those that pushed toward "e" are strengthened. Repeated over many examples, the connections come to encode useful patterns.

Career

Rumelhart was on the faculty of the University of California, San Diego from 1967 to 1987, then returned to Stanford University in 1987.^[1] There he led a team of graduate students and postdoctoral fellows until a neurodegenerative condition took away his intellectual powers in the 1990s. He retired to Ann Arbor, Michigan, in 1998.^[1]

Honours and legacy

His honours include a MacArthur Fellowship (1987), election to the US National Academy of Sciences (1991), the American Psychological Association's Distinguished Scientific Contribution Award, and the 2002 Grawemeyer Award in Psychology, which he shared with McClelland.^[3] In 2000 the David E. Rumelhart Prize for Contributions to the Theoretical Foundations of Cognitive Science was established in his honour.^[3]

The ideas he helped spread have outlasted the systems of his time. Back-propagation became one of the standard ways to train neural networks, and modern deep learning, which uses far larger networks trained on far more data, still adjusts connection strengths in response to errors. The 1986 paper belongs to the same line of work.^[2]