Volcano Engine launches the Doubao model family and cuts LLM prices

Background

Volcano Engine is the cloud computing unit of ByteDance. Companies that build products on large language models pay for each unit of text the model processes, and that unit is the token. A token is a piece of text, often a word or part of a word. Prices are therefore quoted per thousand or per million tokens, and a lower price changes what a business can afford to build.

What happened

On 15 May 2024 Volcano Engine launched the Doubao model family at an event in Beijing.^[1] Tan Dai, the president of Volcano Engine, introduced nine models. They included Pro and Lite editions of general models, speech-recognition models, speech-synthesis models and text-to-image models.^[1]

Key details

The main model was priced at 0.0008 yuan per 1,000 prompt tokens. Caixin said this was 99.8% cheaper than GPT-4.^[2] Tan said comparable enterprise models averaged 0.12 yuan per 1,000 tokens.^[1]

Doubao Pro supported a context window of up to 128,000 tokens. The context window is the amount of text the model can consider at one time, so a longer window lets it work with longer documents.^[2]

At the time of the launch, the Doubao app had more than 100 million downloads and more than 26 million monthly active users.^[2]

Enterprise customers compare models on capability, speed and price. For a company that sends a very large number of requests, a small change in the per-token rate can change the monthly bill by a large amount, so a sharp price cut draws attention from businesses even before they have chosen a model.

A window of 128,000 tokens is large enough to hold a long report in a single request, which is why context length is often compared across models.

Significance

Media described the launch as the start of a price war in China's enterprise large-model market.^[3] A price this low lowers the cost of trying many uses before cost becomes a limit.