Qwen3 Technical Report

The Qwen3 Technical Report is credited to the Qwen Team at Alibaba, with An Yang, Anfeng Li and Baosong Yang leading the author list. It was submitted to arXiv on 14 May 2025 as 2505.09388. The paper describes Qwen3, a family of large language models that spans both dense and mixture-of-experts architectures, with parameter counts from 0.6 billion to 235 billion. The flagship model, Qwen3-235B-A22B, is a mixture-of-experts model.^[1]

A dense model applies one full set of parameters to each token, while a mixture-of-experts model routes work among separate expert subnetworks. Qwen3 covers both designs rather than using one architecture for the whole family.^[1] That choice matters because model size and serving cost are linked: different deployments can use different points on the range.

The central design change is a unified framework that combines a thinking mode for complex multi-step reasoning with a non-thinking mode for fast, context-driven responses. The report says this removes the need to switch between separate chat-optimized models and reasoning-specific models. Mode selection can follow the query or the chat template.^[1] In ordinary use, a chat-optimized model is tuned for quick replies, while a reasoning-specific model spends more computation on intermediate steps; Qwen3 puts both behaviours in one framework.

Qwen3 also adds a thinking budget mechanism. This lets users allocate inference compute adaptively, trading latency against performance according to how complex the task is.^[1] Inference compute is the computation used when a model generates an answer, so a larger budget can allow more work before the response is produced, at the cost of waiting longer. The budget gives an application a single control instead of treating speed and reasoning depth as fixed properties of separate models.

Pre-training used about 36 trillion tokens covering up to 119 languages and dialects, up from 29 in Qwen2.5.^[1] The process followed three stages: about 30 trillion tokens for general knowledge, then knowledge-intensive data to strengthen STEM and coding reasoning, then long-context data.^[1] Pre-training is the broad first phase in which a model learns statistical patterns from large amounts of text; the staged recipe changes what kind of material the model sees later in that phase.

The report says knowledge from the flagship models significantly reduces the compute needed for the smaller models while keeping them competitive.^[1] That is the practical purpose of a family running from 0.6 billion to 235 billion parameters: smaller models can serve constrained settings, while the largest mixture-of-experts model represents the top of the range.

All Qwen3 models are released under the Apache 2.0 licence.^[1] Apache 2.0 is a widely used open-source licence. The release therefore combines open licensing across the full size range with a single framework for thinking and non-thinking operation, plus a way to spend more or less inference compute. The report presents these as the main changes in Qwen3: one family for both fast responses and multi-step reasoning, broader language coverage in pre-training, and a staged data recipe built for general knowledge, STEM and coding, and long context.^[1]