Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 30 · Clear filters
- 2026
Muse Spark Safety & Preparedness Report
arXiv
Meta's safety and preparedness report for Muse Spark, the model behind Meta AI, sets out the catastrophic-risk evaluations behind its launch and argues that, with mitigations applied, residual risk is acceptable.
- 2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
- 2026
GLM-5: from Vibe Coding to Agentic Engineering
arXiv
Zhipu's GLM-5 report describes a 744-billion-parameter mixture-of-experts model built on sparse attention and asynchronous reinforcement learning, aimed at moving AI-assisted programming from vibe coding toward agentic engineering.
- 2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
arXiv
Terminal-Bench 2.0 is a curated benchmark of 89 hard tasks carried out in computer terminal environments, built to measure how well AI agents handle realistic command-line work; frontier models and agents score below 65% on it.
- 2025
Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
arXiv
The Gemini 2.5 technical report, submitted to arXiv on 7 July 2025, documents Google's Gemini 2.5 Pro and 2.5 Flash models: their reasoning, long-context video handling, and cost-versus-capability positioning.
- 2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
- 2024
The Llama 3 Herd of Models
arXiv
Meta's 2024 paper describing Llama 3 as a family of foundation models — 8B, 70B and 405B parameters — trained on about 15 trillion tokens and reported to match leading models such as GPT-4 across a wide range of tasks.
- 2024
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
ICML
Tri Dao and Albert Gu's paper introduces state space duality, a framework showing that structured state space models and attention variants are closely related, and uses it to derive Mamba-2, whose core layer runs 2 to 8 times faster than Mamba's.
- 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
arXiv
A benchmark for code LLMs that keeps collecting fresh contest problems from LeetCode, AtCoder and CodeForces, so that scores reflect ability rather than memorised test data, and that also tests code repair and execution reasoning.
- 2024
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
arXiv
A report from Google's Gemini Team introducing the Gemini 1.5 family, multimodal models built around very long context, with Gemini 1.5 Pro recalling information across up to 10 million tokens.[^1]
- 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
arXiv
A benchmark of 2,294 real software engineering problems taken from GitHub issues and the pull requests that resolved them, in which a model must edit a codebase to fix an issue; on release the best model solved under 2% of them.
- 2022
Scalable Diffusion Models with Transformers (DiT)
arXiv
A 2022 paper by William Peebles and Saining Xie that replaces the U-Net backbone of diffusion models with a transformer operating on latent patches, showing that image quality improves predictably as compute grows and reaching a state-of-the-art FID of 2.27 on ImageNet 256×256.
- 2022
Training Compute-Optimal Large Language Models (Chinchilla)
NeurIPS
A DeepMind study showing that, for a fixed computing budget, language models should be trained on far more data than was then standard; its 70-billion-parameter Chinchilla beat larger contemporaries.
- 2021
Program Synthesis with Large Language Models (MBPP)
arXiv
A 2021 arXiv paper by Austin, Odena, Nye and eight other authors that introduced MBPP, a benchmark of 974 short Python programming tasks, and measured how language models from 244M to 137B parameters turn natural-language descriptions into working code.
- 2021
Evaluating Large Language Models Trained on Code (Codex / HumanEval)
arXiv
A 2021 paper from OpenAI introduces Codex, a GPT model fine-tuned on public GitHub code, and HumanEval, an evaluation set that judges generated Python programs by whether they work rather than by how closely they match a reference.
- 2021
Learning Transferable Visual Models From Natural Language Supervision (CLIP)
ICML
CLIP is an OpenAI model trained on 400 million image-text pairs to map pictures and sentences into one shared space, so it can classify images zero-shot from a written description.
- 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)
arXiv
A 2020 paper showing that a plain Transformer applied directly to sequences of image patches, with no convolution at all, can match state-of-the-art convolutional networks on image classification.
- 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
arXiv 2010.11929 (ICLR 2021)
A 2020 paper by Alexey Dosovitskiy and colleagues showing that a pure Transformer applied directly to sequences of image patches — the Vision Transformer, or ViT — matches state-of-the-art convolutional networks on image classification when pre-trained on large amounts of data.
- 2020
Language Models are Few-Shot Learners (GPT-3)
NeurIPS
The GPT-3 paper showed that a language model with 175 billion parameters could perform many tasks from a few examples in its prompt, with no fine-tuning, and that this ability grew with model size.
- 2020
Scaling Laws for Neural Language Models
arXiv
A 2020 paper finding that language model loss falls predictably, as a power law, with model size, data and compute. It gave labs a quantitative reason to build much larger models and shaped the scale-up that followed.