Qwen3-Next

Background

Qwen3-Next was released in September 2025 under the Apache 2.0 license by Alibaba's Qwen team. It was built to explore a different architecture from the standard transformer layout used by most large language models. Its design combined linear attention with sparse mixture-of-experts layers, and this combination later became the architecture of Qwen3.5 in February 2026. ^[2][1][3]

What it is

Qwen3-Next-80B-A3B has 80 billion total parameters, of which only 3 billion are activated for each token. A token is a unit of text, and activated parameters are those used to process it. The naming reflects this split: 80B is the total size, and A3B indicates about 3B active parameters. ^[1]

Architecture

The network is arranged in a hybrid layout. It has 48 layers grouped into 12 blocks. Each block contains three Gated DeltaNet layers, which are a form of linear attention, followed by one Gated Attention layer. Every one of these layers is followed by a mixture-of-experts (MoE) layer. ^[1]

Standard attention compares every token with every other token, and its cost grows quickly with the length of the text. Linear attention is an approximation that scales more gently with length, which is why hybrid designs mix a small number of full attention layers with many cheaper ones. ^[1]

The MoE part is highly sparse. It has 512 experts, and for each token, 10 routed experts plus 1 shared expert are activated. A routed expert is chosen for each token by a router, while the shared expert handles every token. ^[1]

Training and efficiency

Qwen3-Next was pre-trained on 15 trillion tokens. Alibaba reported that its base model beat Qwen3-32B-Base on downstream tasks while using only 10% of the training cost of that model. ^[1]

The design also targets inference, meaning the work of producing output. The model uses multi-token prediction, a training technique in which the model predicts several upcoming tokens rather than one, which helps pre-training and supports speculative decoding, a method that drafts several tokens quickly and then verifies them. For contexts longer than 32K tokens, the model was reported to deliver 10 times the inference throughput of Qwen3-32B-Base. ^[1]

Context and performance

The native context window is 262,144 tokens, and it can be extended to about one million tokens using YaRN, a technique for stretching a model's context length. ^[1]

The Instruct version was evaluated on several benchmarks. It scored 69.5 on AIME25, slightly below the 70.3 of Qwen3-235B-A22B-Instruct-2507. It scored 82.7 on Arena-Hard v2, against 79.2 for that model, and 56.6 on LiveCodeBench v6, against 51.8. These were the figures reported by Alibaba, and they show a mixed picture rather than a uniform lead. ^[1]

Position in the Qwen family

Qwen3-Next matters because it tested architectural choices that later appeared in the Qwen3.5 generation. The Gated DeltaNet and sparse MoE combination became the basis of that family in February 2026. For a general reader, the main lesson is that a model with a very large total parameter count can be run at the cost of a much smaller model, if most of its parameters stay inactive for each token. ^[3]