Kimi K3: Open Frontier Intelligence

What the report is

On 27 July 2026 the Kimi Team at Moonshot AI submitted a technical report to arXiv, numbered 2607.24653, describing a model called Kimi K3 ^[1]. The full weights of the model were released together with the report ^[1].

K3 is a mixture-of-experts (MoE) model with 2.8 trillion total parameters, of which 104 billion are activated per token; it has native vision capabilities and a context window of one million tokens ^[1]. In an MoE model, the network is divided into many expert subnetworks and a router sends each token to only a few of them, so the total parameter count can be far larger than the amount of computation spent on any single token.

Architecture

K3 builds on two attention-related designs, Kimi Delta Attention and Attention Residuals, which the report says improve the flow of information across sequence length and across model depth ^[1]. Its Stable LatentMoE layer activates 16 of 896 routed experts for each token ^[1].

Together with refined training and data recipes, these changes yield roughly a 2.5x improvement in overall scaling efficiency over the earlier Kimi K2 ^[1]. Scaling efficiency refers to how much capability a model gains per unit of training resource; a 2.5x improvement means a given level of quality is reached with substantially less of that resource than before.

Training and post-training

Post-training puts its emphasis on reinforcement learning across general, agentic and coding domains, and on multiple reasoning-effort levels ^[1]. Reasoning-effort levels are settings that let the same model spend more or less computation on thinking before it answers, trading latency for depth depending on the task.

Serving a 2.8-trillion-parameter model

Running a model of this size required work on the systems side as much as the modelling side. The report lists three pieces: algorithm–system co-design for Kimi Delta Attention; perfectly balanced expert-parallel training with efficient memory management; and million-token agentic reinforcement learning with persistent rollout and sandbox states ^[1]. Expert parallelism splits the experts of an MoE model across many devices, so balancing how much work each device gets is what keeps large-scale training from stalling on its slowest participant.

Results

Evaluations find frontier-level performance on long-horizon coding, agentic, knowledge, reasoning and vision tasks ^[1]. The report is also explicit about the ceiling: overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT-5.6 Sol, while consistently beating the other open and proprietary models in its evaluation suite ^[1].

Why it matters

Two things stand out. The first is scale with open weights: a 2.8-trillion-parameter model with native vision and a million-token context, published with its full weights rather than behind an API ^[1]. The second is the engineering account — the attention redesigns, the expert-parallel training scheme and the long-horizon reinforcement learning setup are described as parts of one system that had to be co-designed to make the model trainable and servable at all ^[1]. Read together with the evaluation caveat, the report presents K3 as a step that closes much of the distance to the strongest closed models without yet overtaking them ^[1].