Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 15 · Clear filters
- 2022
Magnetic control of tokamak plasmas through deep reinforcement learning
Nature
A 2022 Nature paper by Degrave, Felici and colleagues showing that a deep reinforcement learning controller can learn on its own to command all of a tokamak's magnetic control coils, producing and holding a range of plasma shapes.
- 2020
Scaling Laws for Neural Language Models
arXiv
A 2020 paper finding that language model loss falls predictably, as a power law, with model size, data and compute. It gave labs a quantitative reason to build much larger models and shaped the scale-up that followed.
- 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
arXiv 1909.08053
An NVIDIA paper from 2019 presenting an intra-layer model parallelism method, implemented with a few communication operations in native PyTorch, that trained transformer models of up to 8.3 billion parameters and improved several benchmarks.
- 2012
Large Scale Distributed Deep Networks
NeurIPS
A 2012 NIPS paper by Jeffrey Dean, Andrew Ng and colleagues that introduced DistBelief, a framework for training deep networks across clusters of thousands of machines, together with the Downpour SGD and Sandblaster optimisation procedures.
- 2012
ImageNet Classification with Deep Convolutional Neural Networks
NeurIPS
The 2012 NeurIPS paper that introduced AlexNet, a large convolutional network trained on GPUs. It won the ILSVRC-2012 image classification competition by a wide margin and is widely seen as a turning point for deep learning in computer vision.
- 2009
Large-scale Deep Unsupervised Learning using Graphics Processors
ICML
A 2009 ICML paper by Rajat Raina, Anand Madhavan and Andrew Y. Ng that used graphics processors to train deep belief networks and sparse coding models many times faster than multicore CPUs.
- 1982
Neural networks and physical systems with emergent collective computational abilities
PNAS
A 1982 paper by John J. Hopfield arguing that useful computation can emerge as a collective property of many simple, equivalent units, and describing a model whose collective behaviour gives content-addressable memory, generalisation and error correction.
- 1969
Perceptrons: An Introduction to Computational Geometry
A 1969 book by Marvin Minsky and Seymour Papert that proved mathematical limits on what single-layer perceptrons can compute — notably that they cannot compute parity or connectedness — and has long been credited with steering research away from neural networks.
- 1966
ELIZA - A Computer Program for the Study of Natural Language Communication Between Man and Machine
Communications of the ACM 9(1)
A January 1966 paper by Joseph Weizenbaum presenting ELIZA, a program on MIT's MAC time-sharing system that used keyword-triggered decomposition and reassembly rules to enable certain natural-language conversations between people and computers.
- 1961
Steps Toward Artificial Intelligence
Proceedings of the IRE 49(1)
Marvin Minsky's 1961 survey in Proceedings of the IRE organized heuristic programming into five problem areas — search, pattern recognition, learning, planning and induction — and argued that machines would need to build models of their environments.
- 1959
Some Studies in Machine Learning Using the Game of Checkers
IJRD
A. L. Samuel's 1959 paper reporting experiments in which a checkers program, given only the rules of the game and an imperfect set of parameters, learned in 8 to 10 hours of playing time to beat the person who wrote it.
- 1959
Programs with Common Sense
PSMTPNPL
John McCarthy's 1959 paper proposing the advice taker, a program that would solve problems by manipulating sentences in formal languages and act when it deduced an imperative — an early statement of logical AI.
- 1950
Computing Machinery and Intelligence
Mind
In 1950 Alan Turing replaced the question "Can machines think?" with the imitation game, now called the Turing test, and answered the main objections to machine intelligence one by one.
- 1943
A Logical Calculus of the Ideas Immanent in Nervous Activity
Bull. Math. Biophys.
In 1943 Warren McCulloch and Walter Pitts described a simplified model of a neuron and showed that networks of such units can carry out logical reasoning, an early link between brain models and computation.
- 1936
On Computable Numbers, with an Application to the Entscheidungsproblem
Proc. London Math. Soc.
Alan Turing's 1936 paper defined computable numbers through an abstract machine, now called the Turing machine, and showed that no general method can decide whether any given mathematical statement is provable.