Kimi K2: Open Agentic Intelligence

The problem the report addresses

Kimi K2 is a mixture-of-experts (MoE) language model with 1 trillion total parameters, of which 32 billion are activated for a given token^[1]. In an MoE model, each token is routed to a small subset of the network rather than the whole thing, so the model can hold far more parameters than it uses on any single pass. The technical report describing K2 is credited to the Kimi Team at Moonshot AI and was submitted to arXiv on 28 July 2025 as 2507.20534^[1].

Two engineering problems run through the report. The first is stability: training a model of this size tends to produce sudden spikes in the loss, which waste compute and can force a restart. The second is capability of a particular kind — the report is built around agentic behaviour, meaning the model's ability to carry out multi-step work such as calling tools, editing code and operating inside an environment rather than only answering a single question.

Pre-training with MuonClip

The team's answer to the stability problem is an optimizer called MuonClip. It extends the Muon optimizer with a weight-clipping mechanism named QK-Clip^[1]. Muon is valued for its token efficiency — roughly, how much learning it extracts from each token of training data — and the goal of MuonClip is to keep that efficiency while removing the instability. Clipping, in this setting, means putting a bound on the size of certain weights during training so that a small number of very large updates cannot push the model into a bad state; QK-Clip applies that idea at the point in the attention computation where queries and keys meet.

The result reported is that, with MuonClip, K2 was pre-trained on 15.5 trillion tokens with zero loss spikes^[1]. That is the central training claim of the paper: not just a lower loss, but a run of that length that never had to be rescued.

Post-training

After pre-training, K2 goes through a multi-stage post-training procedure^[1]. Its centre is a large-scale agentic data synthesis pipeline — an automated way of producing the kind of training material that teaches a model to act, not merely to answer — combined with a joint reinforcement learning stage in which the model improves through interaction with real and synthetic environments^[1]. Reinforcement learning here means the model is rewarded for behaviour that succeeds in an environment, so the training signal comes from doing tasks rather than from imitating text.

Results

The report describes K2 as state of the art among open-source non-thinking models, with particular strength in agentic capabilities^[1]. The reported agentic scores are 66.1 on Tau2-Bench, 76.5 on ACEBench (English), 65.8 on SWE-Bench Verified and 47.3 on SWE-Bench Multilingual^[1]. On general benchmarks, without extended thinking, it scores 53.7 on LiveCodeBench v6, 49.5 on AIME 2025, 75.1 on GPQA-Diamond and 27.1 on OJBench^[1].

The phrase "non-thinking" matters for reading these numbers: K2 is being compared with open models that answer directly, without a long internal reasoning pass before responding, and the benchmark results above are quoted for that mode.

Why it matters

The report is a description of one model, but its interest is broader. It documents a training recipe — an optimizer change for stability, a very long pre-training run, and a post-training stack organised around agentic data and reinforcement learning — at the trillion-parameter scale^[1]. It also puts concrete numbers on where an open model of this kind stands on agentic benchmarks such as Tau2-Bench and SWE-Bench^[1].

Alongside the report, the team released base and post-trained checkpoints^[1]. That release is what makes the work checkable and reusable: the training claims can be tested against the weights, and the post-trained model can be run on the same benchmarks by anyone else.