Home›Papers
Papers
Papers
Papers that changed AI, with a plain-language explainer for each.
Total 21 · Clear filters
- 2026
Kimi K3: Open Frontier Intelligence
arXiv
Kimi K3 is Moonshot AI's 2.8-trillion-parameter open mixture-of-experts model with native vision and a one-million-token context, released with full weights. Its report describes the architecture, training and serving work behind it, and reports frontier-level results that still trail the strongest proprietary models.
- 2026
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
arXiv
A preview report on the DeepSeek-V4 series, describing a 1.6-trillion-parameter Pro model and a 284-billion-parameter Flash model, both with one-million-token context, plus the attention, connection and optimization changes that make that length affordable.
- 2026
Qwen3.5-Omni Technical Report
arXiv
The Qwen3.5-Omni Technical Report documents Alibaba's Qwen3.5-Omni, a hundreds-of-billions-parameter model that understands long audio and video and generates speech across 10 languages, reporting state-of-the-art results on 215 audio and audio-visual benchmarks.
- 2026
GLM-5: from Vibe Coding to Agentic Engineering
arXiv
Zhipu's GLM-5 report describes a 744-billion-parameter mixture-of-experts model built on sparse attention and asynchronous reinforcement learning, aimed at moving AI-assisted programming from vibe coding toward agentic engineering.
- 2025
Kimi K2: Open Agentic Intelligence
arXiv
Kimi K2 is Moonshot AI's trillion-parameter mixture-of-experts model, pre-trained with the MuonClip optimizer and post-trained on large-scale agentic data plus reinforcement learning; the report reports state-of-the-art results among open-source non-thinking models and releases base and post-trained checkpoints.
- 2025
Qwen3 Technical Report
arXiv
The Qwen3 Technical Report presents Alibaba's Qwen3 family of dense and mixture-of-experts language models, which unify thinking and non-thinking modes, add a controllable thinking budget, and are released under the Apache 2.0 licence.
- 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
arXiv
The DeepSeek-R1 report described how a reasoning model was trained largely with reinforcement learning, rewarding correct answers, and reached performance comparable to OpenAI's o1 on math, code and reasoning benchmarks. Its weights were released openly.
- 2024
DeepSeek-V3 Technical Report
arXiv
A technical report from DeepSeek-AI on DeepSeek-V3, a mixture-of-experts language model with 671 billion total parameters, describing its architecture, training pipeline, and evaluations, including a full training run of 2.788 million H800 GPU hours.
- 2024
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
arXiv
FlashAttention-3 is an attention algorithm for NVIDIA Hopper (H100) GPUs that reaches up to 75% utilization in FP16 and close to 1.2 PFLOPs/s in FP8, using asynchronous execution and low-precision techniques.
- 2023
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
arXiv
FlashAttention-2 is a 2023 revision of the FlashAttention attention kernel that reorganizes work across GPU thread blocks and warps, roughly doubling speed and reaching up to 73% of the A100's theoretical peak throughput.
- 2023
QLoRA: Efficient Finetuning of Quantized LLMs
arXiv
QLoRA is a finetuning method that trains a 65B-parameter language model on a single 48GB GPU while matching full 16-bit finetuning quality, using 4-bit quantization, double quantization and paged optimizers.
- 2022
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
arXiv
FlashAttention is an exact attention algorithm built around GPU memory traffic: by tiling computation so intermediates stay on chip, it cuts reads and writes to high bandwidth memory, training Transformers faster and enabling longer sequences.
- 2021
LoRA: Low-Rank Adaptation of Large Language Models
arXiv
LoRA is a fine-tuning method that freezes a pre-trained model's weights and trains small low-rank matrices injected into each layer, cutting trainable parameters and memory by orders of magnitude.
- 2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
arXiv
GShard, submitted to arXiv in June 2020, is a set of annotation APIs and an XLA compiler extension from Google that lets existing model code express parallel computation, scaling a sparsely gated mixture-of-experts translation model past 600 billion parameters.
- 2017
Proximal Policy Optimization Algorithms
arXiv
A 2017 paper by Schulman, Wolski, Dhariwal, Radford and Klimov that introduced proximal policy optimization (PPO), a reinforcement learning method that is much simpler to implement and tune and became OpenAI's default RL algorithm.
- 2012
Large Scale Distributed Deep Networks
NeurIPS
A 2012 NIPS paper by Jeffrey Dean, Andrew Ng and colleagues that introduced DistBelief, a framework for training deep networks across clusters of thousands of machines, together with the Downpour SGD and Sandblaster optimisation procedures.
- 2011
Torch7: A Matlab-like Environment for Machine Learning
A 2011 workshop paper by Ronan Collobert, Koray Kavukcuoglu and Clement Farabet describing Torch7, a Lua-based numeric computing framework and machine learning library built for flexible experimentation on fast low-level kernels.
- 2010
Understanding the difficulty of training deep feedforward neural networks
AISTATS / PMLR
A 2010 paper by Xavier Glorot and Yoshua Bengio on why deep feedforward networks are hard to train: sigmoid saturation under random initialization and layer Jacobians whose singular values sit far from 1, plus a new initialization scheme that converges substantially faster.
- 2004
MapReduce: Simplified Data Processing on Large Clusters
OSDI
Jeffrey Dean and Sanjay Ghemawat's 2004 OSDI paper presents MapReduce, a programming model in which users write a map function and a reduce function while the run-time system parallelizes the work across large clusters of commodity machines.
- 1990
Elephants Don't Play Chess
Rodney Brooks's 1990 paper argued for building artificial intelligence from the ground up, with ongoing physical interaction with the environment as the main constraint on design, rather than through the abstract manipulation of symbols that had dominated the field for roughly thirty years.