StepFun open-sources the Step-Audio speech models

Background StepFun (阶跃星辰) was working on speech interaction as well as video. Speech interaction means talking to a model by voice and getting spoken answers. On 17 February 2025 the company made this work available to the public.

What happened On 17 February 2025 StepFun published the Step-Audio technical report and released the inference code and weights ^[1]. The technical report is arXiv:2502.11946 ^[2].

The release comprised three components: Step-Audio-Chat, with 130 billion parameters, Step-Audio-TTS-3B and Step-Audio-Tokenizer ^[1]. The tokenizer converts speech into a sequence of tokens that a language model can process, and TTS (text-to-speech) turns text into spoken audio.

Key details The models support Chinese, English and Japanese, as well as dialects including Cantonese and Sichuanese. They can control emotion and singing style ^[1].

StepFun also released the StepEval-Audio-360 benchmark for evaluating these models ^[1].

The release was a joint open-sourcing with Geely, made at the same time as Step-Video-T2V ^[3].

Significance The event made a 130-billion-parameter speech model available with code, weights and a report on one day.

Terms used Speech models come in several kinds. A speech recognition model turns speech into text, a text-to-speech model turns text into speech, and a speech-interaction model does both within a conversation. The tokenizer in this release converts audio into discrete tokens that a language model can work with.

How the parts fit together The three components cover the two sides of a voice conversation. The tokenizer handles audio input, the chat model generates the reply, and the TTS model produces the spoken answer.

What an open release provides An open release lets outside developers download the code and weights, run the models on their own hardware and check the reported results. The licence terms decide what they may do with them. Here the code was under Apache 2.0, and the weights had their own terms in the repositories.