Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 100 · Clear filters
- 2016
Mastering the game of Go with deep neural networks and tree search
Nature
The 2016 Nature paper describing AlphaGo, which combined deep neural networks with tree search. In October 2015 its distributed version beat the European Go champion Fan Hui 5-0, the first time a program defeated a professional player in full-sized Go without handicap.
- 2015
Deep Residual Learning for Image Recognition
CVPR
The 2015 paper introduced residual networks, which add each layer's input back to its output through a shortcut connection. This made networks of over a hundred layers trainable and won first place in several ILSVRC and COCO tasks in 2015.
- 2015
You Only Look Once: Unified, Real-Time Object Detection
arXiv
The 2015 YOLO paper recast object detection as a single neural network's regression over bounding boxes and class probabilities, achieving real-time detection while reducing false positives on empty background.
- 2015
You Only Look Once: Unified, Real-Time Object Detection
arXiv
The 2015 paper introduced YOLO, which treats object detection as a single regression from image pixels to bounding boxes and class probabilities. One neural network makes predictions in one evaluation, reaching real-time speed.
- 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
arXiv
A 2015 paper that introduced a Region Proposal Network sharing full-image convolutional features with the detection network, making proposals nearly cost-free and enabling end-to-end training, 5 fps GPU detection, and state-of-the-art accuracy.
- 2015
Human-level control through deep reinforcement learning
Nature
A 2015 Nature paper by Mnih, Kavukcuoglu, Silver and colleagues showed that one deep Q-network agent, given only screen pixels and the game score, learned to play 49 Atari 2600 games at a level comparable to a professional human tester.
- 2015
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
arXiv
A 2015 paper by Sergey Ioffe and Christian Szegedy that proposed normalising each layer's inputs per training mini-batch, which allowed higher learning rates, looser initialisation and much faster training of deep networks.
- 2015
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
arXiv
A 2015 arXiv paper by Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho and co-authors that introduced an image-captioning model with visual attention, able to show where it is looking while writing each word.
- 2015
Deep learning
Nature
A 2015 Nature review by Yann LeCun, Yoshua Bengio and Geoffrey Hinton explaining deep learning: models of multiple processing layers that learn layered representations of data, and the results this produced in speech, vision and other fields.
- 2014
Adam: A Method for Stochastic Optimization
arXiv (ICLR 2015)
Adam is a stochastic optimisation algorithm from a 2014 paper by Diederik P. Kingma and Jimmy Ba that sets per-parameter step sizes from running estimates of the gradient's first and second moments, needing little memory and little hyper-parameter tuning.
- 2014
Deep Speech: Scaling up end-to-end speech recognition
arXiv
A 2014 paper presenting an end-to-end deep-learning speech recognition system that removes hand-engineered noise modules and the phoneme dictionary, relying instead on recurrent networks trained across multiple GPUs and synthesized data.
- 2014
Show and Tell: A Neural Image Caption Generator
arXiv
A 2014 paper by Vinyals and colleagues that trained a single deep network to turn a photograph into a sentence, reporting a BLEU-1 score of 59 on Pascal against 25 for the previous state of the art.
- 2014
Conditional Generative Adversarial Nets
arXiv
A 2014 paper by Mehdi Mirza and Simon Osindero that adds conditioning data to both the generator and the discriminator, enabling a GAN to generate samples such as MNIST digits conditioned on class labels.
- 2014
Neural Turing Machines
arXiv
A 2014 paper by Graves, Wayne and Danihelka that couples a neural network to an external memory it reads and writes through attention, making the whole machine differentiable and trainable by gradient descent.
- 2014
GloVe: Global Vectors for Word Representation
EMNLP 2014
GloVe is a 2014 method that learns word vectors by factorising global word co-occurrence counts. Released together with pre-trained vectors, it became, alongside word2vec, one of the two standard ways of producing static word embeddings.
- 2014
Going Deeper with Convolutions (GoogLeNet / Inception)
arXiv
A 2014 paper from Google researchers that introduced the Inception architecture, whose GoogLeNet model set a new state of the art in ILSVRC 2014 by widening and deepening a network within a fixed computational budget.
- 2014
Going Deeper with Convolutions
arXiv / CVPR 2015
The 2014 paper that introduced the Inception architecture and its 22-layer GoogLeNet network, which took first place in the ImageNet 2014 classification and detection challenge with a 6.67% top-5 error.
- 2014
Sequence to Sequence Learning with Neural Networks
NeurIPS
Ilya Sutskever, Oriol Vinyals and Quoc Le's 2014 paper showed that two LSTMs, one reading the input and one writing the output, could translate sentences and outperform a phrase-based statistical baseline. It set the encoder–decoder pattern behind later translation models.
- 2014
Very Deep Convolutional Networks for Large-Scale Image Recognition
arXiv / ICLR 2015
A 2014 study showing that stacking many small 3×3 convolution filters, up to 16–19 weight layers, improves large-scale image recognition; its models took first place in localisation and second in classification at ImageNet Challenge 2014 and were released publicly.
- 2014
Neural Machine Translation by Jointly Learning to Align and Translate
ICLR
Submitted to arXiv in September 2014 by Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio, this paper let a translation model look back at the whole source sentence while producing each word. This mechanism, called attention, later became a basis for Transformer models.