Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 116 · Clear filters
- 2017
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Science
A 2017 arXiv paper, later published in Science, reporting that a single self-play reinforcement learning algorithm, AlphaZero, mastered chess, shogi and Go and beat strong specialised programs in each.
- 2017
Progressive Growing of GANs for Improved Quality, Stability, and Variation
arXiv
A 2017 paper by Karras, Aila, Laine and Lehtinen that trains generative adversarial networks by growing the generator and discriminator from low to high resolution, improving stability, quality and variation.
- 2017
mixup: Beyond Empirical Risk Minimization
arXiv
mixup is a training principle from a 2017 paper by Zhang, Cisse, Dauphin and Lopez-Paz that trains networks on convex combinations of pairs of examples and their labels, improving generalisation on image, speech and tabular benchmarks.
- 2017
Mastering the game of Go without human knowledge
Nature
AlphaGo Zero learned Go only by playing against itself, starting from random play, and beat the earlier champion-defeating AlphaGo 100 games to 0 after three days of training. The Nature paper appeared in October 2017.
- 2017
Focal Loss for Dense Object Detection
arXiv
A 2017 paper by Lin, Goyal, Girshick, He and Dollar that argues dense one-stage detectors are held back by extreme foreground-background imbalance and proposes focal loss, a reshaped cross-entropy that down-weights well-classified examples.
- 2017
Proximal Policy Optimization Algorithms
arXiv
A 2017 paper by Schulman, Wolski, Dhariwal, Radford and Klimov that introduced proximal policy optimization (PPO), a reinforcement learning method that is much simpler to implement and tune and became OpenAI's default RL algorithm.
- 2017
Attention Is All You Need
NeurIPS
A 2017 paper that introduced the Transformer, a neural network built entirely on attention with no recurrence. It set new machine translation results and became the architecture behind almost every large language model since.
- 2017
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
arXiv
CycleGAN learns image-to-image translation between two domains without aligned pairs, combining an adversarial loss with a cycle-consistency constraint to constrain the mapping.
- 2017
Mask R-CNN
arXiv
Mask R-CNN is a 2017 framework for object instance segmentation. It detects objects and produces a high-quality mask for each instance by adding a parallel mask branch to Faster R-CNN.
- 2016
Neural Architecture Search with Reinforcement Learning
arXiv
A 2016 paper by Barret Zoph and Quoc V. Le that uses a recurrent network trained with reinforcement learning to design neural network architectures automatically, matching human-designed models on CIFAR-10 and beating LSTM on Penn Treebank.
- 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
arXiv
A 2016 paper describing Google's neural machine translation system, GNMT: a deep LSTM encoder-decoder with attention, designed to make end-to-end translation practical by addressing rare words, parallelism and inference cost.
- 2016
WaveNet: A Generative Model for Raw Audio
arXiv
WaveNet is a deep neural network that generates raw audio waveforms one sample at a time, predicting each sample from all the ones before it. Used for text-to-speech, it was rated by listeners as more natural than the systems it was compared with.
- 2016
Identity Mappings in Deep Residual Networks
arXiv
A 2016 analysis of residual networks arguing that with identity skip connections and the activation placed after addition, signals travel directly between blocks; a new residual unit built on this eases training, and a 1001-layer network reaches 4.62% error on CIFAR-10.
- 2016
Mastering the game of Go with deep neural networks and tree search
Nature
The 2016 Nature paper describing AlphaGo, which combined deep neural networks with tree search. In October 2015 its distributed version beat the European Go champion Fan Hui 5-0, the first time a program defeated a professional player in full-sized Go without handicap.
- 2015
Deep Residual Learning for Image Recognition
CVPR
The 2015 paper introduced residual networks, which add each layer's input back to its output through a shortcut connection. This made networks of over a hundred layers trainable and won first place in several ILSVRC and COCO tasks in 2015.
- 2015
You Only Look Once: Unified, Real-Time Object Detection
arXiv
The 2015 YOLO paper recast object detection as a single neural network's regression over bounding boxes and class probabilities, achieving real-time detection while reducing false positives on empty background.
- 2015
You Only Look Once: Unified, Real-Time Object Detection
arXiv
The 2015 paper introduced YOLO, which treats object detection as a single regression from image pixels to bounding boxes and class probabilities. One neural network makes predictions in one evaluation, reaching real-time speed.
- 2015
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
arXiv
A 2015 paper that introduced a Region Proposal Network sharing full-image convolutional features with the detection network, making proposals nearly cost-free and enabling end-to-end training, 5 fps GPU detection, and state-of-the-art accuracy.
- 2015
U-Net: Convolutional Networks for Biomedical Image Segmentation
arXiv
A convolutional network for biomedical image segmentation, posted to arXiv on May 18, 2015, that trains end to end from very few annotated images and won the ISBI cell tracking challenge 2015 in its categories.
- 2015
Human-level control through deep reinforcement learning
Nature
A 2015 Nature paper by Mnih, Kavukcuoglu, Silver and colleagues showed that one deep Q-network agent, given only screen pixels and the game score, learned to play 49 Atari 2600 games at a level comparable to a professional human tester.