Baichuan releases its first open model, Baichuan-7B

Background

Baichuan released Baichuan-7B in June 2023, about two months after the company was founded. The release was an open-source one: the model's weights and documentation were made available for others to download and use, a common practice for language models at the time that lets outside developers inspect, run and build on a model rather than only accessing it through an interface.^[1]

What the model is

Baichuan-7B is a language model with 7 billion parameters, the numerical weights that the model learns during training and that shape its behaviour. It was trained on about 1.2 trillion tokens, the units of text that a model reads during training. The model is bilingual, handling both Chinese and English.^[2]

Its stated context length, the amount of text the model can take in at once, is 4,096 tokens according to the model card on GitHub.^[2]

Results

Two benchmarks were reported for the model, both run in the 5-shot setting, meaning the model was given five examples of a task before being asked to perform it. Baichuan-7B scored 42.8 on C-Eval, a benchmark for evaluating Chinese-language models, and 42.3 on MMLU, a widely used English-language benchmark covering many academic subjects.^[2] On the hard subset of C-Eval, its average was 31.5.^[2]

Access and licensing

Baichuan-7B is not unrestricted: the company allowed commercial use after registration and written authorisation. That arrangement, sometimes called a permissive-but-gated licence, lets researchers and companies use the model while keeping the developer informed about who is using it commercially.^[2]

Why it matters

A 7-billion-parameter open model released two months after a company's founding places Baichuan-7B in the first wave of Chinese open-weight language models, alongside other releases of the period. Publishing an open model with a documented model card, benchmark numbers and an explicit licence gives outside developers a concrete baseline they can test and compare against, and it gives the company a public reference point for later models. Scoring in the low 40s on both a Chinese and an English benchmark also set a visible marker of how a smaller open model of this size performed on bilingual tasks at the time.^[2]