Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 63 · Clear filters
- 2022
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding (Imagen)
arXiv
Imagen is a text-to-image diffusion model that uses a large pretrained language model to encode prompts; it reached a zero-shot FID of 7.27 on COCO and introduced the DrawBench benchmark.
- 2022
Magnetic control of tokamak plasmas through deep reinforcement learning
Nature
A 2022 Nature paper by Degrave, Felici and colleagues showing that a deep reinforcement learning controller can learn on its own to command all of a tokamak's magnetic control coils, producing and holding a range of plasma shapes.
- 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
NeurIPS
A 2022 paper by Jason Wei and colleagues at Google Research showing that prompting a large language model with examples that include intermediate reasoning steps, rather than answers alone, substantially improves its performance on arithmetic, commonsense and symbolic reasoning tasks.
- 2021
High-Resolution Image Synthesis with Latent Diffusion Models
arXiv 2112.10752 (CVPR 2022)
A 2021 paper by Rombach, Blattmann, Lorenz, Esser and Ommer that runs diffusion models inside the latent space of pretrained autoencoders, cutting the compute needed for high-resolution image synthesis while adding cross-attention for text and bounding-box control.
- 2021
Training Verifiers to Solve Math Word Problems (GSM8K)
arXiv
GSM8K is an OpenAI benchmark of 8.5K grade school math word problems, released with a paper showing that even the largest Transformer models scored poorly, and that training verifiers to select among many candidate solutions improved results.
- 2021
Finetuned Language Models Are Zero-Shot Learners
arXiv 2109.01652
A 2021 paper showing that finetuning a language model on many tasks described by instructions — instruction tuning — makes it much better at unseen tasks without examples, surpassing a larger model on most benchmarks tested.
- 2021
Program Synthesis with Large Language Models (MBPP)
arXiv
A 2021 arXiv paper by Austin, Odena, Nye and eight other authors that introduced MBPP, a benchmark of 974 short Python programming tasks, and measured how language models from 244M to 137B parameters turn natural-language descriptions into working code.
- 2021
Evaluating Large Language Models Trained on Code (Codex / HumanEval)
arXiv
A 2021 paper from OpenAI introduces Codex, a GPT model fine-tuned on public GitHub code, and HumanEval, an evaluation set that judges generated Python programs by whether they work rather than by how closely they match a reference.
- 2021
Measuring Mathematical Problem Solving With the MATH Dataset
arXiv
MATH is a dataset of 12,500 challenging competition mathematics problems, each with a full step-by-step solution. Released in 2021, it was built to measure and train mathematical reasoning in models.
- 2021
Zero-Shot Text-to-Image Generation (DALL·E)
arXiv
A 2021 OpenAI paper describing a single 12-billion-parameter autoregressive transformer trained on 250 million image-text pairs, which generates images from text and, evaluated zero-shot, is competitive with domain-specific models.
- 2020
Measuring Massive Multitask Language Understanding (MMLU)
arXiv
A benchmark released in 2020 that tests language models across 57 subjects, from elementary mathematics to law, and showed that even the largest models of the time were far from expert-level.
- 2020
Denoising Diffusion Probabilistic Models
NeurIPS
A 2020 paper by Jonathan Ho, Ajay Jain and Pieter Abbeel that trained a U-Net to reverse a gradual noising process, making diffusion models a practical rival to GANs and paving the way for DALL-E 2, Imagen and Stable Diffusion.
- 2020
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
arXiv
A 2020 paper by Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi and Ng that represents a scene as a neural radiance field: a network mapping 5D coordinates to density and colour, optimized from a sparse set of images with known camera poses.
- 2019
Mastering Atari, Go, chess and shogi by planning with a learned model
Nature
MuZero is a DeepMind reinforcement-learning algorithm that learns a model of its environment and plans inside that model, mastering Atari, Go, chess and shogi without being given the rules of those games.
- 2019
On the Measure of Intelligence (ARC-AGI)
arXiv
A 2019 paper by Francois Chollet that defines intelligence as skill-acquisition efficiency and introduces the Abstraction and Reasoning Corpus (ARC), a benchmark of grid tasks that assumes no specialised knowledge or language.
- 2019
HellaSwag: Can a Machine Really Finish Your Sentence?
arXiv
HellaSwag is a 2019 benchmark in which a machine must pick the most plausible continuation of a short event description. Its wrong answers are generated and filtered to fool models rather than people.
- 2018
A Style-Based Generator Architecture for Generative Adversarial Networks
arXiv
This 2018 paper introduces a style-based GAN generator that separates high-level attributes from stochastic variation and allows scale-specific control, improving distribution quality, interpolation, and disentanglement.
- 2018
Large Scale GAN Training for High Fidelity Natural Image Synthesis
arXiv
A 2018 paper by Andrew Brock, Jeff Donahue and Karen Simonyan that trained generative adversarial networks at the largest scale attempted at the time, introduced BigGANs and set a new state of the art in class-conditional image synthesis.
- 2018
Glow: Generative Flow with Invertible 1x1 Convolutions
arXiv
A 2018 paper by Diederik P. Kingma and Prafulla Dhariwal presenting Glow, a flow-based generative model built on invertible 1x1 convolutions, which improved log-likelihood on standard benchmarks and could synthesise and manipulate large images.
- 2017
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Science
A 2017 arXiv paper, later published in Science, reporting that a single self-play reinforcement learning algorithm, AlphaZero, mastered chess, shogi and Go and beat strong specialised programs in each.