Baichuan releases Baichuan 3, a hundred-billion-parameter model

Background

Baichuan 3 is a language model from Baichuan, a Chinese AI company. It was released as a closed model: its weights were not published for download, and the company offered it for trial through its own website.

In this entry, a closed model means one whose weights are kept by the developer, so outside users cannot download and run it themselves.

What happened

On 29 January 2024 Baichuan released Baichuan 3, a language model with more than one hundred billion parameters. ^[1] The model was offered for trial at baichuan-ai.com. ^[1]

At launch, Baichuan said the model surpassed GPT-4 on Chinese benchmarks such as CMMLU, a Chinese multi-subject knowledge test, and GAOKAO, which uses Chinese college entrance exam questions. ^[1] These were the company's own evaluation results, not independent tests.

Key details

Baichuan also said the model beat GPT-4 on Chinese medical exams, including MCMLE, MedExam and CMExam. ^[1] Its pre-training data included more than one hundred billion tokens of medical text. A token is a small piece of text, often a word or part of a word. ^[1]

The model was trained with RLHF and RLAIF in an iterative reinforcement-learning process. RLHF uses human feedback to guide a model toward answers people prefer, and RLAIF uses feedback produced by AI. ^[1]

The comparison names need explaining. GPT-4 and GPT-3.5 are language models from OpenAI, and they are often used as reference points when a new model is compared with others. When a company says a model surpasses a named model, the claim applies only to the tests the company chose to report, under the conditions it describes.

Impact

The launch put the company's focus on Chinese-language tests and medical applications in public view. Because the figures come from Baichuan's own evaluation, readers should treat them as the company's claims. The company's later medical work is described in the Baichuan-M entries.