Baichuan 2

What it is

Baichuan 2 is Baichuan's second-generation family of open models. It was released on 6 September 2023 in two sizes, 7B and 13B. ^[1] Each size has a Base version and a Chat version, and the company also released 4-bit quantised Chat models. ^[2]

The Base version is the model after pre-training on large amounts of text. The Chat version is trained further to answer questions and hold a conversation. A 4-bit quantised model stores its numbers with four bits instead of the usual sixteen, which makes it much smaller to run.

Origins

The family continues the first generation of Baichuan's open models, which included Baichuan-7B. ^[1] Baichuan released the models together with a public code repository, a technical report and a set of research materials. ^[2][3]

Key features

Baichuan 2 was trained on 2.6 trillion tokens. ^[2] A token is a small piece of text, often a word or part of a word, so the figure gives a sense of the amount of text the models read during training.

Quantisation changes how much hardware is needed. Baichuan reported that 4-bit quantisation cut the GPU memory needed by the 13B model from 27.5GB to 8.6GB. ^[2]

Baichuan also released 11 intermediate checkpoints of the 7B model, taken at training points from about 0.2 to 2.4 trillion tokens, for research. A checkpoint is a saved copy of the model during training, which lets researchers see how it changes as it learns. ^[2]

In late December 2023 the company released Baichuan2-13B-Chat v2, on 29 December, which improved maths, logical reasoning and the following of complex instructions. ^[2]

Benchmark results

Benchmarks are standard tests used to compare models. Baichuan reported the following 5-shot results for Baichuan2-13B-Base: 58.10 on C-Eval, a Chinese exam benchmark; 59.17 on MMLU, a multi-subject English test; and 61.97 on CMMLU, a Chinese version of that test. It reported 52.77 on GSM8K, a grade-school maths test with 4-shot prompting, and 17.07 on HumanEval, a code-writing test with 0-shot prompting. ^[2]

For the 7B Base model, Baichuan reported 54.00 on C-Eval, 54.16 on MMLU and 57.07 on CMMLU. ^[2] These are the company's own figures, as published in its technical report.

Licence

The models were released under Apache 2.0 together with the Baichuan 2 Model Community License. Commercial use required an application. It was limited to services with fewer than 1 million daily active users, and to companies that are not software or cloud providers. ^[2]

Position in development

Baichuan 2 is the point at which the company's open models moved to a second generation, with two sizes, quantised versions, training checkpoints and a published technical report released together. The technical report, 'Baichuan 2: Open Large-scale Language Models', is arXiv:2309.10305. ^[3] Wikipedia notes a Baichuan2 version whose context window could handle about 350,000 Chinese characters. ^[4]