Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 32 · Clear filters
- 2026
Kimi K3: Open Frontier Intelligence
arXiv
Kimi K3 is Moonshot AI's 2.8-trillion-parameter open mixture-of-experts model with native vision and a one-million-token context, released with full weights. Its report describes the architecture, training and serving work behind it, and reports frontier-level results that still trail the strongest proprietary models.
- 2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
- 2026
Qwen3.5-Omni Technical Report
arXiv
The Qwen3.5-Omni Technical Report documents Alibaba's Qwen3.5-Omni, a hundreds-of-billions-parameter model that understands long audio and video and generates speech across 10 languages, reporting state-of-the-art results on 215 audio and audio-visual benchmarks.
- 2026
GLM-5: from Vibe Coding to Agentic Engineering
arXiv
Zhipu's GLM-5 report describes a 744-billion-parameter mixture-of-experts model built on sparse attention and asynchronous reinforcement learning, aimed at moving AI-assisted programming from vibe coding toward agentic engineering.
- 2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
arXiv
Terminal-Bench 2.0 is a curated benchmark of 89 hard tasks carried out in computer terminal environments, built to measure how well AI agents handle realistic command-line work; frontier models and agents score below 65% on it.
- 2025
Kimi K2: Open Agentic Intelligence
arXiv
Kimi K2 is Moonshot AI's trillion-parameter mixture-of-experts model, pre-trained with the MuonClip optimizer and post-trained on large-scale agentic data plus reinforcement learning; the report reports state-of-the-art results among open-source non-thinking models and releases base and post-trained checkpoints.
- 2025
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
arXiv
A 2025 paper introducing ARC-AGI-2, a benchmark of input-output puzzle tasks built to measure abstract reasoning in frontier AI systems, with human testing as a baseline and a task set designed to give a more granular signal of progress.
- 2025
Qwen3 Technical Report
arXiv
The Qwen3 Technical Report presents Alibaba's Qwen3 family of dense and mixture-of-experts language models, which unify thinking and non-thinking modes, add a controllable thinking budget, and are released under the Apache 2.0 licence.
- 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
arXiv
The DeepSeek-R1 report described how a reasoning model was trained largely with reinforcement learning, rewarding correct answers, and reached performance comparable to OpenAI's o1 on math, code and reasoning benchmarks. Its weights were released openly.
- 2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
- 2024
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
ICML
Tri Dao and Albert Gu's paper introduces state space duality, a framework showing that structured state space models and attention variants are closely related, and uses it to derive Mamba-2, whose core layer runs 2 to 8 times faster than Mamba's.
- 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
arXiv
A benchmark for code LLMs that keeps collecting fresh contest problems from LeetCode, AtCoder and CodeForces, so that scores reflect ability rather than memorised test data, and that also tests code repair and execution reasoning.
- 2023
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
arXiv
A benchmark of 2,294 real software engineering problems taken from GitHub issues and the pull requests that resolved them, in which a model must edit a codebase to fix an issue; on release the best model solved under 2% of them.
- 2022
Learning skillful medium-range global weather forecasting
Science
GraphCast is a graph neural network trained on four decades of reanalysis data that produces 10-day global weather forecasts at 0.25 degree resolution in under a minute, outperforming the leading operational deterministic systems on most verification targets.
- 2022
Scalable Diffusion Models with Transformers (DiT)
arXiv
A 2022 paper by William Peebles and Saining Xie that replaces the U-Net backbone of diffusion models with a transformer operating on latent patches, showing that image quality improves predictably as compute grows and reaching a state-of-the-art FID of 2.27 on ImageNet 256×256.
- 2022
Magnetic control of tokamak plasmas through deep reinforcement learning
Nature
A 2022 Nature paper by Degrave, Felici and colleagues showing that a deep reinforcement learning controller can learn on its own to command all of a tokamak's magnetic control coils, producing and holding a range of plasma shapes.
- 2021
Program Synthesis with Large Language Models (MBPP)
arXiv
A 2021 arXiv paper by Austin, Odena, Nye and eight other authors that introduced MBPP, a benchmark of 974 short Python programming tasks, and measured how language models from 244M to 137B parameters turn natural-language descriptions into working code.
- 2021
Evaluating Large Language Models Trained on Code (Codex / HumanEval)
arXiv
A 2021 paper from OpenAI introduces Codex, a GPT model fine-tuned on public GitHub code, and HumanEval, an evaluation set that judges generated Python programs by whether they work rather than by how closely they match a reference.
- 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)
arXiv
A 2020 paper showing that a plain Transformer applied directly to sequences of image patches, with no convolution at all, can match state-of-the-art convolutional networks on image classification.
- 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
arXiv 2010.11929 (ICLR 2021)
A 2020 paper by Alexey Dosovitskiy and colleagues showing that a pure Transformer applied directly to sequences of image patches — the Vision Transformer, or ViT — matches state-of-the-art convolutional networks on image classification when pre-trained on large amounts of data.