Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 57 · Clear filters
- 2010
Understanding the difficulty of training deep feedforward neural networks
AISTATS / PMLR
A 2010 paper by Xavier Glorot and Yoshua Bengio on why deep feedforward networks are hard to train: sigmoid saturation under random initialization and layer Jacobians whose singular values sit far from 1, plus a new initialization scheme that converges substantially faster.
- 2009
Large-scale Deep Unsupervised Learning using Graphics Processors
ICML
A 2009 ICML paper by Rajat Raina, Anand Madhavan and Andrew Y. Ng that used graphics processors to train deep belief networks and sparse coding models many times faster than multicore CPUs.
- 2003
Latent Dirichlet Allocation
JMLR
A generative probabilistic model that represents each document as a finite mixture over topics, fitted with variational approximate inference and EM, and evaluated on document modelling, text classification and collaborative filtering.
- 1999
The PageRank Citation Ranking: Bringing Order to the Web
Brin and Page's PageRank technical report treats hyperlinks as citations to rank web pages and was used to build Google's search engine; it is one of the heavily cited papers on their early work.
- 1998
The Anatomy of a Large-Scale Hypertextual Web Search Engine
A 1998 conference paper by Sergey Brin and Lawrence Page describing PageRank, an algorithm that ranks web pages by the number and quality of links pointing to them, and the search engine built on that algorithm.
- 1992
A Training Algorithm for Optimal Margin Classifiers
Published in 1992, this paper by Boser, Guyon and Vapnik presents a training algorithm that maximizes the margin between training patterns and the decision boundary, with experiments on optical character recognition.[^1]
- 1992
Practical issues in temporal difference learning (TD-Gammon)
Machine Learning
Gerald Tesauro's 1992 paper tested whether temporal difference learning could master a complex real-world task, using self-play backgammon as the case study.
- 1990
Elephants Don't Play Chess
Rodney Brooks's 1990 paper argued for building artificial intelligence from the ground up, with ongoing physical interaction with the environment as the main constraint on design, rather than through the abstract manipulation of symbols that had dominated the field for roughly thirty years.
- 1988
Learning to predict by the methods of temporal differences
Machine Learning
Richard Sutton's 1988 paper introduced temporal-difference learning, a family of incremental prediction methods that assign credit from changes between successive predictions and are shown to be more economical and accurate than conventional supervised approaches on most prediction problems.
- 1986
Learning representations by back-propagating errors
Nature
In a 1986 Nature paper, Rumelhart, Hinton and Williams showed that backpropagation lets hidden units in a multilayer network learn useful internal features of a task. The method is standard in most neural networks today.
- 1985
A Learning Algorithm for Boltzmann Machines
Cognitive Science 9
A 1985 Cognitive Science paper by Ackley, Hinton and Sejnowski that presents a general learning rule for connectionist networks and reports simple examples of efficient internal representations.
- 1982
Neural networks and physical systems with emergent collective computational abilities
PNAS
A 1982 paper by John J. Hopfield showing that large networks of simple neuron-like units can collectively store and recall memories. It became known as the Hopfield network, and the work was recognised by the 2024 Nobel Prize in Physics.
- 1982
Neural networks and physical systems with emergent collective computational abilities
PNAS
A 1982 paper by John J. Hopfield arguing that useful computation can emerge as a collective property of many simple, equivalent units, and describing a model whose collective behaviour gives content-addressable memory, generalisation and error correction.
- 1969
Perceptrons: An Introduction to Computational Geometry
A 1969 book by Marvin Minsky and Seymour Papert that proved mathematical limits on what single-layer perceptrons can compute — notably that they cannot compute parity or connectedness — and has long been credited with steering research away from neural networks.
- 1966
ELIZA - A Computer Program for the Study of Natural Language Communication Between Man and Machine
Communications of the ACM 9(1)
A January 1966 paper by Joseph Weizenbaum presenting ELIZA, a program on MIT's MAC time-sharing system that used keyword-triggered decomposition and reassembly rules to enable certain natural-language conversations between people and computers.
- 1959
Some Studies in Machine Learning Using the Game of Checkers
IJRD
A. L. Samuel's 1959 paper reporting experiments in which a checkers program, given only the rules of the game and an imperfect set of parameters, learned in 8 to 10 hours of playing time to beat the person who wrote it.
- 1948
A Mathematical Theory of Communication
Bell System Technical Journal 27
C. E. Shannon's 1948 paper set out a mathematical framework for communication, defining information in bits, channel capacity, and entropy, and treating messages statistically while setting aside semantic meaning.