Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 34 · Clear filters
- 2018
Glow: Generative Flow with Invertible 1x1 Convolutions
arXiv
A 2018 paper by Diederik P. Kingma and Prafulla Dhariwal presenting Glow, a flow-based generative model built on invertible 1x1 convolutions, which improved log-likelihood on standard benchmarks and could synthesise and manipulate large images.
- 2017
Neural Discrete Representation Learning
arXiv
A 2017 paper by van den Oord, Vinyals and Kavukcuoglu proposing VQ-VAE, a generative model whose latent code is discrete rather than continuous, and whose prior is learned rather than fixed, so that useful representations can be learned without supervision.
- 2017
Progressive Growing of GANs for Improved Quality, Stability, and Variation
arXiv
A 2017 paper by Karras, Aila, Laine and Lehtinen that trains generative adversarial networks by growing the generator and discriminator from low to high resolution, improving stability, quality and variation.
- 2017
Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks
arXiv
CycleGAN learns image-to-image translation between two domains without aligned pairs, combining an adversarial loss with a cycle-consistency constraint to constrain the mapping.
- 2016
WaveNet: A Generative Model for Raw Audio
arXiv
WaveNet is a deep neural network that generates raw audio waveforms one sample at a time, predicting each sample from all the ones before it. Used for text-to-speech, it was rated by listeners as more natural than the systems it was compared with.
- 2014
Generative Adversarial Nets
arXiv
Goodfellow, Bengio, and co-authors proposed a generative-model framework with adversarially trained generator and discriminator.
- 2013
Auto-Encoding Variational Bayes
arXiv
A 2013 paper by Diederik P. Kingma and Max Welling that introduced a stochastic variational inference and learning algorithm for directed latent-variable models with intractable posteriors, making approximate inference practical on large datasets.
- 2011
Torch7: A Matlab-like Environment for Machine Learning
A 2011 workshop paper by Ronan Collobert, Koray Kavukcuoglu and Clement Farabet describing Torch7, a Lua-based numeric computing framework and machine learning library built for flexible experimentation on fast low-level kernels.
- 2010
Phone recognition with the mean-covariance restricted Boltzmann Machine
NIPS
A 2010 NIPS paper by Dahl, Ranzato, Mohamed and Hinton applies Deep Belief Nets and a mean-covariance RBM to acoustic modeling, reaching 20.5% phone error on speaker-independent TIMIT phone recognition.
- 2010
Understanding the difficulty of training deep feedforward neural networks
AISTATS / PMLR
A 2010 paper by Xavier Glorot and Yoshua Bengio on why deep feedforward networks are hard to train: sigmoid saturation under random initialization and layer Jacobians whose singular values sit far from 1, plus a new initialization scheme that converges substantially faster.
- 2006
A fast learning algorithm for deep belief nets
Neural Computation
A 2006 paper by Hinton, Osindero and Teh that uses complementary priors and a greedy layer-by-layer algorithm to train deep belief nets, then fine-tunes them; the resulting three-hidden-layer model both modelled and classified handwritten digits well.
- 2004
MapReduce: Simplified Data Processing on Large Clusters
OSDI
Jeffrey Dean and Sanjay Ghemawat's 2004 OSDI paper presents MapReduce, a programming model in which users write a map function and a reduce function while the run-time system parallelizes the work across large clusters of commodity machines.
- 2003
Latent Dirichlet Allocation
JMLR
A generative probabilistic model that represents each document as a finite mixture over topics, fitted with variational approximate inference and EM, and evaluated on document modelling, text classification and collaborative filtering.
- 1985
A Learning Algorithm for Boltzmann Machines
Cognitive Science 9
A 1985 Cognitive Science paper by Ackley, Hinton and Sejnowski that presents a general learning rule for connectionist networks and reports simple examples of efficient internal representations.