Richard Sutton

Overview

Richard Sutton is a Professor in Computing Science at the University of Alberta. He is also a Research Scientist at Keen Technologies, an artificial general intelligence company based in Dallas, Texas, and Chief Scientific Advisor of the Alberta Machine Intelligence Institute (Amii).^[1] His research centres on reinforcement learning, the branch of machine learning in which an agent learns by acting in an environment and adjusting its behaviour in response to the rewards it receives, rather than by being shown labelled examples of correct answers. That distinction, between learning from feedback and learning from instruction, runs through most of his work: the questions he has pursued are about how a system can improve from its own experience when nobody tells it what it should have done.

Education and early research

Sutton received his BA in Psychology from Stanford University, and later earned his MS and PhD degrees in Computer and Information Science from the University of Massachusetts at Amherst.^[1] The combination of psychology and computer science is unusual, and it shows in the shape of the field he helped build. Reinforcement learning borrows much of its vocabulary from the study of animal behaviour, such as reward, trial and error, and learning from experience, while its methods come from computer science, statistics and optimisation. Someone trained in both has an easier time asking why a learning rule should look the way it does, not just whether it converges.

His collaboration with Andrew Barto began in 1978 at the University of Massachusetts at Amherst, where Barto was Sutton's PhD and postdoctoral advisor.^[1] The partnership that started there lasted for decades and produced both foundational research in the area and the textbook that introduced generations of students to it.

Main contributions

The most visible product of that collaboration is *Reinforcement Learning: An Introduction*, a textbook the two co-authored in 1998.^[1] The book laid out the ideas of the field in a single framework: an agent that observes something about its situation, takes actions, receives rewards, and forms estimates of how much reward it can expect from the situations it encounters. It remains the standard reference in the field and has been cited more than 75,000 times.^[1]

What makes that framework useful is that it holds together problems that at first look unrelated. A program learning to play a board game, a robot learning to walk, and a system allocating resources across a network can all be described in the same terms, and the same questions arise in each case. One is the trade-off between exploring unfamiliar actions and exploiting the actions already known to work, since a learner who never tries anything new cannot discover a better strategy, but a learner who never settles on anything good wastes what it has learned. Another is how to assign credit: when a reward arrives long after the actions that caused it, which of those actions should be reinforced? The textbook organised these questions, and the algorithms devised to answer them, into a common language.

Career

Before joining the University of Alberta, Sutton served as a Principal Technical Staff Member in the Artificial Intelligence Department at the AT&T Shannon Laboratory in Florham Park, New Jersey, from 1998 to 2002.^[1] He was a Distinguished Research Scientist at DeepMind from 2017 to 2023.^[1]

His honors include the IJCAI Research Excellence Award and a Lifetime Achievement Award from the Canadian Artificial Intelligence Association. He is a Fellow of the Royal Society of London, of the Association for the Advancement of Artificial Intelligence, and of the Royal Society of Canada.^[1]

Impact

In 2024 Sutton was named a recipient of the ACM A.M. Turing Award together with Andrew Barto.^[1] The Turing Award is widely described as the highest distinction in computing, and the recognition came at a time when reinforcement learning had become a standard tool, used wherever a system has to make sequences of decisions and learn from the results rather than match a fixed set of answers. For a field that spent years as a specialised corner of machine learning, the award marked how central its questions had become: how an agent should act under uncertainty, how it should balance exploration against what it already knows, and how it should learn from delayed feedback. The textbook Sutton wrote with Barto, and the students he supervised and taught, carry that framework onward.