Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 23 · Clear filters
- 2025
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
arXiv
A 2025 paper introducing ARC-AGI-2, a benchmark of input-output puzzle tasks built to measure abstract reasoning in frontier AI systems, with human testing as a baseline and a task set designed to give a more granular signal of progress.
- 2024
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
arXiv
FlashAttention-3 is an attention algorithm for NVIDIA Hopper (H100) GPUs that reaches up to 75% utilization in FP16 and close to 1.2 PFLOPs/s in FP8, using asynchronous execution and low-precision techniques.
- 2024
Accurate structure prediction of biomolecular interactions with AlphaFold 3
Nature
AlphaFold 3, published in Nature in May 2024, extends structure prediction to biomolecular interactions, reporting greater accuracy than docking tools for protein–ligand binding and than AlphaFold-Multimer v.2.3 for antibody–antigen complexes.
- 2023
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
arXiv
FlashAttention-2 is a 2023 revision of the FlashAttention attention kernel that reorganizes work across GPU thread blocks and warps, roughly doubling speed and reaching up to 73% of the A100's theoretical peak throughput.
- 2023
QLoRA: Efficient Finetuning of Quantized LLMs
arXiv
QLoRA is a finetuning method that trains a 65B-parameter language model on a single 48GB GPU while matching full 16-bit finetuning quality, using 4-bit quantization, double quantization and paged optimizers.
- 2022
Learning skillful medium-range global weather forecasting
Science
GraphCast is a graph neural network trained on four decades of reanalysis data that produces 10-day global weather forecasts at 0.25 degree resolution in under a minute, outperforming the leading operational deterministic systems on most verification targets.
- 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
arXiv
FlashAttention is an exact attention algorithm built around GPU memory traffic: by tiling computation so intermediates stay on chip, it cuts reads and writes to high bandwidth memory, training Transformers faster and enabling longer sequences.
- 2022
Magnetic control of tokamak plasmas through deep reinforcement learning
Nature
A 2022 Nature paper by Degrave, Felici and colleagues showing that a deep reinforcement learning controller can learn on its own to command all of a tokamak's magnetic control coils, producing and holding a range of plasma shapes.
- 2021
Highly accurate protein structure prediction with AlphaFold
Nature
The Nature paper described AlphaFold's methods, system design, and evaluation for protein-structure prediction.
- 2021
LoRA: Low-Rank Adaptation of Large Language Models
arXiv
LoRA is a fine-tuning method that freezes a pre-trained model's weights and trains small low-rank matrices injected into each layer, cutting trainable parameters and memory by orders of magnitude.
- 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
arXiv
GShard, submitted to arXiv in June 2020, is a set of annotation APIs and an XLA compiler extension from Google that lets existing model code express parallel computation, scaling a sparsely gated mixture-of-experts translation model past 600 billion parameters.
- 2020
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
arXiv
A 2020 paper by Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi and Ng that represents a scene as a neural radiance field: a network mapping 5D coordinates to density and colour, optimized from a sparse set of images with known camera poses.
- 2019
Mastering Atari, Go, chess and shogi by planning with a learned model
Nature
MuZero is a DeepMind reinforcement-learning algorithm that learns a model of its environment and plans inside that model, mastering Atari, Go, chess and shogi without being given the rules of those games.
- 2019
On the Measure of Intelligence (ARC-AGI)
arXiv
A 2019 paper by Francois Chollet that defines intelligence as skill-acquisition efficiency and introduces the Abstraction and Reasoning Corpus (ARC), a benchmark of grid tasks that assumes no specialised knowledge or language.
- 2017
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Science
A 2017 arXiv paper, later published in Science, reporting that a single self-play reinforcement learning algorithm, AlphaZero, mastered chess, shogi and Go and beat strong specialised programs in each.
- 2017
Proximal Policy Optimization Algorithms
arXiv
A 2017 paper by Schulman, Wolski, Dhariwal, Radford and Klimov that introduced proximal policy optimization (PPO), a reinforcement learning method that is much simpler to implement and tune and became OpenAI's default RL algorithm.
- 2015
U-Net: Convolutional Networks for Biomedical Image Segmentation
arXiv
A convolutional network for biomedical image segmentation, posted to arXiv on May 18, 2015, that trains end to end from very few annotated images and won the ISBI cell tracking challenge 2015 in its categories.
- 2012
Large Scale Distributed Deep Networks
NeurIPS
A 2012 NIPS paper by Jeffrey Dean, Andrew Ng and colleagues that introduced DistBelief, a framework for training deep networks across clusters of thousands of machines, together with the Downpour SGD and Sandblaster optimisation procedures.
- 2011
Torch7: A Matlab-like Environment for Machine Learning
A 2011 workshop paper by Ronan Collobert, Koray Kavukcuoglu and Clement Farabet describing Torch7, a Lua-based numeric computing framework and machine learning library built for flexible experimentation on fast low-level kernels.
- 2010
Understanding the difficulty of training deep feedforward neural networks
AISTATS / PMLR
A 2010 paper by Xavier Glorot and Yoshua Bengio on why deep feedforward networks are hard to train: sigmoid saturation under random initialization and layer Jacobians whose singular values sit far from 1, plus a new initialization scheme that converges substantially faster.