Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 45 · Clear filters
- 2026
GLM-5: from Vibe Coding to Agentic Engineering
arXiv
Zhipu's GLM-5 report describes a 744-billion-parameter mixture-of-experts model built on sparse attention and asynchronous reinforcement learning, aimed at moving AI-assisted programming from vibe coding toward agentic engineering.
- 2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
arXiv
Terminal-Bench 2.0 is a curated benchmark of 89 hard tasks carried out in computer terminal environments, built to measure how well AI agents handle realistic command-line work; frontier models and agents score below 65% on it.
- 2025
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
arXiv
An eight-terabyte collection of public domain and openly licensed text for pretraining language models, released together with two 7B-parameter models trained on it to show the data is enough.
- 2025
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
arXiv
A 2025 paper introducing ARC-AGI-2, a benchmark of input-output puzzle tasks built to measure abstract reasoning in frontier AI systems, with human testing as a baseline and a task set designed to give a more granular signal of progress.
- 2024
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
arXiv
FineWeb is a 15-trillion-token pretraining dataset built from 96 Common Crawl snapshots and released in 2024 with its curation code and ablation models; its educational subset, FineWeb-Edu, improves results on knowledge- and reasoning-heavy benchmarks.
- 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
arXiv
A benchmark for code LLMs that keeps collecting fresh contest problems from LeetCode, AtCoder and CodeForces, so that scores reflect ability rather than memorised test data, and that also tests code repair and execution reasoning.
- 2024
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
arXiv
Dolma is an openly released English corpus of three trillion tokens, built from web text, papers, code, books, social media and Wikipedia, with a paper documenting both the dataset and the pipeline used to curate it.
- 2023
Scaling deep learning for materials discovery
Nature
A 2023 Nature paper from researchers at Google DeepMind presenting a scaled-up deep-learning approach to predicting stable crystal structures, screening 2.2 million candidates and reporting a large jump in stability-prediction accuracy.
- 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
arXiv
A benchmark of 2,294 real software engineering problems taken from GitHub issues and the pull requests that resolved them, in which a model must edit a codebase to fix an issue; on release the best model solved under 2% of them.
- 2023
Segment Anything
arXiv 2304.02643
A 2023 paper that introduces the Segment Anything project: a promptable segmentation task, a model, and a dataset of over 1 billion masks on 11 million images, all released to advance foundation models for computer vision.
- 2023
RedPajama: open reproduction of the LLaMA training dataset
Together AI blog / GitHub
RedPajama is an open-source project that began by reproducing the LLaMA training dataset, releasing a 1.2-trillion-token corpus as a clean-room, fully open implementation.
- 2022
Learning skillful medium-range global weather forecasting
Science
GraphCast is a graph neural network trained on four decades of reanalysis data that produces 10-day global weather forecasts at 0.25 degree resolution in under a minute, outperforming the leading operational deterministic systems on most verification targets.
- 2022
Holistic Evaluation of Language Models (HELM)
arXiv
A 2022 paper by Percy Liang, Rishi Bommasani, Tony Lee and many co-authors that ran 30 language models through the same 42 scenarios and seven metrics, turning scattered benchmarks into one standardized evaluation.
- 2022
LAION-5B: An open large-scale dataset for training next generation image-text models
arXiv
LAION-5B is an openly available dataset of 5.85 billion CLIP-filtered image-text pairs, released so that researchers outside a handful of labs could study and reproduce large vision-language models.
- 2022
Training Compute-Optimal Large Language Models (Chinchilla)
NeurIPS
A DeepMind study showing that, for a fixed computing budget, language models should be trained on far more data than was then standard; its 70-billion-parameter Chinchilla beat larger contemporaries.
- 2022
Magnetic control of tokamak plasmas through deep reinforcement learning
Nature
A 2022 Nature paper by Degrave, Felici and colleagues showing that a deep reinforcement learning controller can learn on its own to command all of a tokamak's magnetic control coils, producing and holding a range of plasma shapes.
- 2021
Program Synthesis with Large Language Models (MBPP)
arXiv
A 2021 arXiv paper by Austin, Odena, Nye and eight other authors that introduced MBPP, a benchmark of 974 short Python programming tasks, and measured how language models from 244M to 137B parameters turn natural-language descriptions into working code.
- 2021
Evaluating Large Language Models Trained on Code (Codex / HumanEval)
arXiv
A 2021 paper from OpenAI introduces Codex, a GPT model fine-tuned on public GitHub code, and HumanEval, an evaluation set that judges generated Python programs by whether they work rather than by how closely they match a reference.
- 2020
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
arXiv
The Pile is an 825 GiB English text dataset of 22 combined sources, released by EleutherAI on 31 December 2020 with an arXiv paper. It became the training data for several of the group's open models.
- 2020
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
arXiv
A 2020 paper by Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi and Ng that represents a scene as a neural radiance field: a network mapping 5D coordinates to density and colour, optimized from a sparse set of images with known camera poses.