Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 85 · Clear filters
- 2010
Phone recognition with the mean-covariance restricted Boltzmann Machine
NIPS
A 2010 NIPS paper by Dahl, Ranzato, Mohamed and Hinton applies Deep Belief Nets and a mean-covariance RBM to acoustic modeling, reaching 20.5% phone error on speaker-independent TIMIT phone recognition.
- 2010
Understanding the difficulty of training deep feedforward neural networks
AISTATS / PMLR
A 2010 paper by Xavier Glorot and Yoshua Bengio on why deep feedforward networks are hard to train: sigmoid saturation under random initialization and layer Jacobians whose singular values sit far from 1, plus a new initialization scheme that converges substantially faster.
- 2009
ImageNet: A Large-Scale Hierarchical Image Database
CVPR
The 2009 paper introduced ImageNet, a large, cleanly labelled image database organised by the WordNet hierarchy. It later became the basis of an annual benchmark, the ImageNet Large Scale Visual Recognition Challenge.
- 2009
Learning Multiple Layers of Features from Tiny Images
UT
A 2009 technical report by Alex Krizhevsky that introduces CIFAR-10 and CIFAR-100 and studies multi-layer generative models for learning features from tiny colour images.
- 2009
Learning Multiple Layers of Features from Tiny Images (CIFAR-10/100)
UT
CIFAR-10 and CIFAR-100 are labeled image datasets drawn from the 80 million tiny images collection, documented in a 2009 technical report by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton. CIFAR-10 holds 60,000 32×32 colour images in 10 classes.
- 2003
Latent Dirichlet Allocation
JMLR
A generative probabilistic model that represents each document as a finite mixture over topics, fitted with variational approximate inference and EM, and evaluated on document modelling, text classification and collaborative filtering.
- 1999
The PageRank Citation Ranking: Bringing Order to the Web
Brin and Page's PageRank technical report treats hyperlinks as citations to rank web pages and was used to build Google's search engine; it is one of the heavily cited papers on their early work.
- 1998
Gradient-Based Learning Applied to Document Recognition
Proc. IEEE
LeCun, Bottou, Bengio and Haffner’s 1998 review argued that learning systems should replace hand-designed features. It showed that convolutional networks such as LeNet-5 outperformed other methods on handwritten digits, and it introduced the MNIST dataset.
- 1998
The Anatomy of a Large-Scale Hypertextual Web Search Engine
A 1998 conference paper by Sergey Brin and Lawrence Page describing PageRank, an algorithm that ranks web pages by the number and quality of links pointing to them, and the search engine built on that algorithm.
- 1998
MNIST database of handwritten digits
Proceedings of the IEEE
MNIST is a database of 60,000 training and 10,000 test images of handwritten digits, normalized to 28×28 pixels and described in the 1998 paper by LeCun and colleagues as a shared benchmark for document recognition.
- 1992
A Training Algorithm for Optimal Margin Classifiers
Published in 1992, this paper by Boser, Guyon and Vapnik presents a training algorithm that maximizes the margin between training patterns and the decision boundary, with experiments on optical character recognition.[^1]
- 1992
Practical issues in temporal difference learning (TD-Gammon)
Machine Learning
Gerald Tesauro's 1992 paper tested whether temporal difference learning could master a complex real-world task, using self-play backgammon as the case study.
- 1990
Elephants Don't Play Chess
Rodney Brooks's 1990 paper argued for building artificial intelligence from the ground up, with ongoing physical interaction with the environment as the main constraint on design, rather than through the abstract manipulation of symbols that had dominated the field for roughly thirty years.
- 1989
Backpropagation Applied to Handwritten Zip Code Recognition
Neural Computation
In 1989 LeCun and colleagues at AT&T Bell Laboratories trained a convolutional network with backpropagation to read handwritten zip-code digits from U.S. mail, reaching 5.0% test error. The work showed that the convolutional design could be learned end to end on a real task.
- 1988
Learning to predict by the methods of temporal differences
Machine Learning
Richard Sutton's 1988 paper introduced temporal-difference learning, a family of incremental prediction methods that assign credit from changes between successive predictions and are shown to be more economical and accurate than conventional supervised approaches on most prediction problems.
- 1988
ALVINN: An Autonomous Land Vehicle in a Neural Network
ANIPS11
A 3-layer back-propagation network that takes camera and laser range finder images and outputs the direction to steer, trained on simulated road images and tested on Carnegie Mellon's autonomous navigation test vehicle.
- 1986
Learning representations by back-propagating errors
Nature
In a 1986 Nature paper, Rumelhart, Hinton and Williams showed that backpropagation lets hidden units in a multilayer network learn useful internal features of a task. The method is standard in most neural networks today.
- 1985
A Learning Algorithm for Boltzmann Machines
Cognitive Science 9
A 1985 Cognitive Science paper by Ackley, Hinton and Sejnowski that presents a general learning rule for connectionist networks and reports simple examples of efficient internal representations.
- 1982
Neural networks and physical systems with emergent collective computational abilities
PNAS
A 1982 paper by John J. Hopfield showing that large networks of simple neuron-like units can collectively store and recall memories. It became known as the Hopfield network, and the work was recognised by the 2024 Nobel Prize in Physics.
- 1982
Neural networks and physical systems with emergent collective computational abilities
PNAS
A 1982 paper by John J. Hopfield arguing that useful computation can emerge as a collective property of many simple, equivalent units, and describing a model whose collective behaviour gives content-addressable memory, generalisation and error correction.