Baichuan 4

What it is

Baichuan 4 is a large language model released by Baichuan, a Chinese AI company, in Beijing on 22 May 2024.^[1][2] The company released it as a closed model, meaning outside developers cannot take the model itself and run it on their own machines.^[1][2] It is also the company's first multimodal model, built to handle inputs beyond plain text.^[5]

Where it came from

Baichuan 4 follows Baichuan 3, which the company had released in late January 2024, only a few months earlier.^[3] Baichuan presented the new model as a step up on that base rather than a change of direction. Compared with Baichuan 3, the company reported general ability up more than 10%, mathematics up 14% and code up 9%.^[2] In a second set of figures it put the gains at 20% in instruction following, 16% in creative writing, 15% in knowledge question answering and 15% in logical reasoning.^[5]

Benchmark claims at launch

On the day of release Baichuan said the model ranked first on the Chinese SuperCLUE benchmark, ahead of GPT-4 Turbo and Claude 3 Opus.^[1] SuperCLUE is an evaluation suite for Chinese-language models: a fixed set of test items is scored and summed into a total, which makes results easy to line up side by side. Baichuan 4 received a total of 80.64 on it, 0.61 points ahead of the previous top model.^[5] On the Chinese comprehensive test it narrowly beat GPT-4-Turbo-0125, by 1.51 points.^[5]

The company also said that on Chinese-language tasks such as knowledge encyclopedia question answering, long-text generation and creative writing, Baichuan 4 clearly outperforms major foreign models, but it did not publish the figures behind that statement.^[3] These are launch-time comparisons; a benchmark score reflects the questions in the suite rather than everything a model does in use.

Launched with an assistant and APIs

The model did not arrive on its own. It powers Baixiaoying, Baichuan's first AI assistant, which launched the same day.^[1] Baichuan 4 was released alongside four APIs: Baichuan 4, Baichuan3-Turbo, Baichuan3-Turbo-128k and an Assistant API.^[5] An API is the interface through which other software calls a model, so developers can put its capabilities into their own products without running the model themselves.

Where it sits in the line

Baichuan 3 and Baichuan 4 were separated by less than half a year, but the company said it planned to slow its model update cadence to roughly once a quarter.^[3] Wikipedia likewise lists Baichuan 4 as released in May 2024.^[4]