ByteDance releases Doubao-1.5-pro

What it is

On 22 January 2025 ByteDance released Doubao-1.5-pro, a closed multimodal model. Closed means the model's weights are not published for others to download. Multimodal means it works with more than one kind of input, such as text and images.^[1][2]

How it was built

Doubao-1.5-pro uses a sparse mixture-of-experts architecture. In this design the model contains many specialist sub-networks, called experts, but for each input only a few of them are switched on. That keeps the cost of running the model lower than if every part worked on every input. ByteDance said this design gave a 7x performance leverage over dense models, where every part of the network is used for every input. It said the industry norm was under 3x.^[1]

ByteDance also said that no data from other models was used in post-training. Post-training is the stage after the first large training run, where the model is refined for particular behaviours.^[1]

Deep-thinking variant

A deep-thinking version, named Doubao-1.5-pro-AS1-Preview, was reported to beat OpenAI o1-preview on AIME, a set of hard competition mathematics problems.^[1]

Pricing and margins

The 32k edition, Doubao-1.5-pro-32k, was priced at 0.8 yuan per million input tokens and 2 yuan per million output tokens.^[2] Jiemian reported, citing an insider, that the model earned a gross margin of 50% on Volcano Engine.^[2]

Mixture-of-experts designs are often described as a way to keep a large model cheaper to run than its total size suggests, because each request uses only part of the network. Multimodal models are judged on how well they combine different kinds of input, not only on text. A deep-thinking version reasons at length before it answers, which usually takes more time and computing power than a direct reply.

Significance

Taken together, the reported pricing and margin describe a model priced low that was still reported to earn a profit.