Step 3

What it is Step 3 is a multimodal reasoning model from StepFun (阶跃星辰). It has 321 billion total parameters, of which 316 billion belong to the language model. It activates 38 billion parameters for each token, the unit of text that a model processes ^[1]. The architecture is Mixture-of-Experts (MoE, 混合专家), so only part of the network works on any given token.

Origins StepFun released Step 3 in July 2025 during the World Artificial Intelligence Conference (WAIC, 世界人工智能大会) ^[3]. Its system technical report, titled "Step-3 is Large yet Affordable", was posted to arXiv on 25 July 2025 ^[3]. The code and weights were released under Apache 2.0 ^[1]. Chinese Wikipedia dates the open-source release to 31 July 2025 ^[5].

At WAIC, StepFun also formed a Model-Chip Ecosystem Innovation Alliance with Huawei (华为), Biren Technology (壁仞科技), Moore Threads (摩尔线程) and Enflame (燧原科技). The alliance aims to optimise Step 3 for domestic chips ^[4].

Main features The model has 61 layers, of which 5 are dense. Its hidden size is 7168. Each MoE layer has 48 experts, with 3 selected for each token and 1 shared expert that every token uses ^[1].

Step 3 uses Multi-Matrix Factorization Attention (MFA), designed together with Attention-FFN Disaggregation (AFD), to reduce the cost of decoding, the step in which the model generates its output one token at a time ^[1]. Disaggregation means splitting the work of the attention and feed-forward parts across hardware, so that each can be scheduled for its own cost profile.

The maximum context length is 65,536 tokens ^[1].

StepFun reported decoding throughput of up to 4,039 tokens per second per GPU, under a 50 ms time-per-output-token limit. The test used a 4K context, FP8 precision and no multi-token prediction (MTP). Under the same setup on Hopper GPUs, DeepSeek-V3 reached 2,324 tokens per second, so Step 3 was about 1.7 times faster ^[3].

The weights are published in bf16 and block-FP8 formats on Hugging Face and ModelScope ^[1].

The model is available through StepFun's open platform, which offers OpenAI-compatible and Anthropic-compatible APIs. It can be deployed with vLLM and SGLang, two common open-source inference engines ^[1][2].

Place in the field Step 3 pairs an open release under Apache 2.0 with a published system report on its design choices, and the chip alliance extends its optimisation work to domestic hardware.