Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 177 · Clear filters
- 2026
Kimi K3: Open Frontier Intelligence
arXiv
Kimi K3 is Moonshot AI's 2.8-trillion-parameter open mixture-of-experts model with native vision and a one-million-token context, released with full weights. Its report describes the architecture, training and serving work behind it, and reports frontier-level results that still trail the strongest proprietary models.
- 2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
- 2026
Qwen3.5-Omni Technical Report
arXiv
The Qwen3.5-Omni Technical Report documents Alibaba's Qwen3.5-Omni, a hundreds-of-billions-parameter model that understands long audio and video and generates speech across 10 languages, reporting state-of-the-art results on 215 audio and audio-visual benchmarks.
- 2026
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
arXiv
Terminal-Bench 2.0 is a curated benchmark of 89 hard tasks carried out in computer terminal environments, built to measure how well AI agents handle realistic command-line work; frontier models and agents score below 65% on it.
- 2025
Kimi K2: Open Agentic Intelligence
arXiv
Kimi K2 is Moonshot AI's trillion-parameter mixture-of-experts model, pre-trained with the MuonClip optimizer and post-trained on large-scale agentic data plus reinforcement learning; the report reports state-of-the-art results among open-source non-thinking models and releases base and post-trained checkpoints.
- 2025
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
arXiv
An eight-terabyte collection of public domain and openly licensed text for pretraining language models, released together with two 7B-parameter models trained on it to show the data is enough.
- 2025
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
arXiv
A 2025 paper introducing ARC-AGI-2, a benchmark of input-output puzzle tasks built to measure abstract reasoning in frontier AI systems, with human testing as a baseline and a task set designed to give a more granular signal of progress.
- 2025
Qwen3 Technical Report
arXiv
The Qwen3 Technical Report presents Alibaba's Qwen3 family of dense and mixture-of-experts language models, which unify thinking and non-thinking modes, add a controllable thinking budget, and are released under the Apache 2.0 licence.
- 2025
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
arXiv
A 2025 paper proposing Absolute Zero, a reinforcement-learning paradigm in which one model writes its own code-reasoning tasks and then solves them, with a code executor as the only judge, reaching state of the art on coding and maths without external training data.
- 2025
s1: Simple test-time scaling
arXiv
A paper showing that a 32B model fine-tuned on 1,000 carefully chosen questions and run with a simple "budget forcing" trick can beat o1-preview on competition maths by up to 27%, with model, data and code released.
- 2025
Humanity's Last Exam (HLE)
arXiv
Humanity's Last Exam (HLE) is a 2,500-question, multi-modal benchmark spanning dozens of academic subjects, built by experts worldwide so that frontier language models can still be measured after saturating older tests.
- 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
arXiv
The DeepSeek-R1 report described how a reasoning model was trained largely with reinforcement learning, rewarding correct answers, and reached performance comparable to OpenAI's o1 on math, code and reasoning benchmarks. Its weights were released openly.
- 2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
- 2024
FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
arXiv
FrontierMath is a benchmark of hundreds of original, unpublished mathematics problems written and vetted by expert mathematicians, introduced by Epoch AI to test advanced mathematical reasoning in AI. At release, state-of-the-art models solved under 2% of the problems.[^1][^2]
- 2024
SAM 2: Segment Anything in Images and Videos
arXiv
SAM 2 is a Meta FAIR foundation model that segments objects in images and videos from simple prompts such as points, boxes or masks, released together with a large video segmentation dataset and training code.
- 2024
The Llama 3 Herd of Models
arXiv
Meta's 2024 paper describing Llama 3 as a family of foundation models — 8B, 70B and 405B parameters — trained on about 15 trillion tokens and reported to match leading models such as GPT-4 across a wide range of tasks.
- 2024
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
arXiv
FlashAttention-3 is an attention algorithm for NVIDIA Hopper (H100) GPUs that reaches up to 75% utilization in FP16 and close to 1.2 PFLOPs/s in FP8, using asynchronous execution and low-precision techniques.
- 2024
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
arXiv
FineWeb is a 15-trillion-token pretraining dataset built from 96 Common Crawl snapshots and released in 2024 with its curation code and ablation models; its educational subset, FineWeb-Edu, improves results on knowledge- and reasoning-heavy benchmarks.
- 2024
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
arXiv
τ-bench is a benchmark, submitted to arXiv in June 2024, that tests language agents in dynamic conversations with a simulated user, with domain-specific API tools and policy rules, and scores them by database state rather than by the text of their replies.
- 2024
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
arXiv
MMLU-Pro is a 2024 benchmark that makes the widely used MMLU knowledge test harder: its questions are reasoning-focused and each has ten answer options instead of four.