Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 75 · Clear filters
- 2014
Microsoft COCO: Common Objects in Context
arXiv
A 2014 paper introducing Microsoft COCO, a large dataset of everyday scenes in which common objects are labeled with per-instance segmentations, built to study object recognition as part of scene understanding.
- 2014
Dropout: A Simple Way to Prevent Neural Networks from Overfitting
JMLR
A 2014 paper by Srivastava, Hinton, Krizhevsky, Sutskever and Salakhutdinov introducing dropout, a training trick that randomly removes units so large neural networks overfit less.
- 2013
Distributed Representations of Words and Phrases and their Compositionality
arXiv
A 2013 paper by Mikolov, Sutskever, Chen, Corrado and Dean presents continuous Skip-gram as an efficient method for learning distributed word and phrase representations, with extensions that improve vector quality and training speed.
- 2013
Speech Recognition with Deep Recurrent Neural Networks
arXiv
A 2013 paper by Graves, Mohamed and Hinton that examines deep recurrent neural networks for speech recognition, arguing for end-to-end sequence training with methods such as connectionist temporal classification.
- 2013
Maxout Networks
arXiv
A 2013 paper by Goodfellow, Warde-Farley, Mirza, Courville and Bengio introduces maxout units, whose output is the maximum of a set of inputs, designed to work with dropout, and reports state-of-the-art results on four benchmarks.
- 2012
Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
IEEE Signal Processing Magazine
A 2012 article in IEEE Signal Processing Magazine in which four research groups argued that deep neural networks, replacing Gaussian mixture models, gave better acoustic models for speech recognition.
- 2011
Deep Sparse Rectifier Neural Networks
A 2011 paper by Glorot, Bordes and Bengio reporting that deep networks built from rectifying neurons match or beat tanh networks and reach their best performance without unsupervised pre-training on large labelled supervised tasks.
- 2011
Natural Language Processing (Almost) from Scratch
A 2011 Journal of Machine Learning Research paper by Collobert, Weston, Bottou, Karlen, Kavukcuoglu and Kuksa, presenting one neural network architecture and learning algorithm for part-of-speech tagging, chunking, named entity recognition and semantic role labeling, trained on mostly unlabeled data with little task-specific engineering.
- 2011
Real-Time Human Pose Recognition in Parts from a Single Depth Image
A CVPR 2011 Best Paper that recovers 3D body joint positions from a single depth image without temporal information, by classifying pixels into body parts, and runs at 200 frames per second on consumer hardware.
- 2010
Phone recognition with the mean-covariance restricted Boltzmann Machine
NIPS
A 2010 NIPS paper by Dahl, Ranzato, Mohamed and Hinton applies Deep Belief Nets and a mean-covariance RBM to acoustic modeling, reaching 20.5% phone error on speaker-independent TIMIT phone recognition.
- 2010
Understanding the difficulty of training deep feedforward neural networks
AISTATS / PMLR
A 2010 paper by Xavier Glorot and Yoshua Bengio on why deep feedforward networks are hard to train: sigmoid saturation under random initialization and layer Jacobians whose singular values sit far from 1, plus a new initialization scheme that converges substantially faster.
- 2009
ImageNet: A Large-Scale Hierarchical Image Database
CVPR
The 2009 paper introduced ImageNet, a large, cleanly labelled image database organised by the WordNet hierarchy. It later became the basis of an annual benchmark, the ImageNet Large Scale Visual Recognition Challenge.
- 2009
Learning Multiple Layers of Features from Tiny Images
UT
A 2009 technical report by Alex Krizhevsky that introduces CIFAR-10 and CIFAR-100 and studies multi-layer generative models for learning features from tiny colour images.
- 2004
MapReduce: Simplified Data Processing on Large Clusters
OSDI
Jeffrey Dean and Sanjay Ghemawat's 2004 OSDI paper presents MapReduce, a programming model in which users write a map function and a reduce function while the run-time system parallelizes the work across large clusters of commodity machines.
- 2003
Latent Dirichlet Allocation
JMLR
A generative probabilistic model that represents each document as a finite mixture over topics, fitted with variational approximate inference and EM, and evaluated on document modelling, text classification and collaborative filtering.
- 2001
Random Forests
Machine Learning
Leo Breiman's 2001 paper defines random forests as combinations of tree predictors whose random vectors are sampled independently from the same distribution, analyzes generalization error through tree strength and correlation, and reports error rates competitive with Adaboost and more robust to noise.
- 1999
The PageRank Citation Ranking: Bringing Order to the Web
Brin and Page's PageRank technical report treats hyperlinks as citations to rank web pages and was used to build Google's search engine; it is one of the heavily cited papers on their early work.
- 1998
Gradient-Based Learning Applied to Document Recognition
Proc. IEEE
LeCun, Bottou, Bengio and Haffner’s 1998 review argued that learning systems should replace hand-designed features. It showed that convolutional networks such as LeNet-5 outperformed other methods on handwritten digits, and it introduced the MNIST dataset.
- 1998
The Anatomy of a Large-Scale Hypertextual Web Search Engine
A 1998 conference paper by Sergey Brin and Lawrence Page describing PageRank, an algorithm that ranks web pages by the number and quality of links pointing to them, and the search engine built on that algorithm.
- 1995
Support-vector networks
Machine Learning
Corinna Cortes and Vladimir Vapnik's 1995 paper presents a learning machine for two-group classification that maps inputs nonlinearly into a high-dimensional feature space, constructs a linear decision surface, and extends earlier work from separable to non-separable data.