Baichuan-M1

What it is

Baichuan-M1 is the medical model line from Baichuan, the Chinese AI company. It was released on 24 January 2025 together with Baichuan-M1-preview, a model Baichuan called China's first all-scenario deep-thinking model. ^[4] The open part of the release is Baichuan-M1-14B, a smaller model available in Base and Instruct versions. ^[4][5][1]

Base and Instruct versions differ in training stage. A base model is trained to continue text, while an instruct model is further tuned to follow requests and answer questions in a conversational way.

Origins

Baichuan-M1-14B is the small open version of Baichuan-M1-preview. Baichuan said its medical ability exceeded that of the larger Qwen2.5-72B and came close to o1-mini. ^[5] Baichuan described it as the industry's first open-source large language model built from scratch for medical scenarios, and as its first open medical model. ^[2]

Key features

Baichuan said it trained the model from scratch on 20 trillion tokens of medical and general data. The medical part included tens of millions of professional medical records, hundreds of millions of medical question-and-answer and clinical items, and nearly a trillion tokens of synthetic medical reasoning data. ^[1] A token is a small piece of text, and synthetic data is text generated by a model for training. The model was tuned for more than 20 medical departments. ^[2]

The architecture includes several changes. It uses short-convolution attention when computing keys and values, which are the parts of the attention mechanism that decide what each word relates to. Some layers use sliding-window attention, which looks only at a nearby stretch of text, to cut the memory used by the KV cache, the stored record of earlier tokens. Some attention heads have larger dimensions, which Baichuan said reduces oscillation in its RoPE positional encoding. ^[1]

On Baichuan's medical benchmark suite the model averaged 72.23. For comparison, Baichuan reported 65.39 for Qwen2.5-14B-Instruct, 70.51 for Qwen2.5-72B-Instruct, 74.85 for Claude 3.5 Sonnet and 75.00 for GPT-4o. ^[1] Baichuan also said it matches similar-sized models on general benchmarks, and that it models five times its size in medical scenarios. ^[2]

Licence and access

Baichuan-M1-14B was released under the Baichuan-M1-14B Model Community License. Commercial use requires contacting Baichuan. ^[2]

Position in development

The technical report, arXiv:2502.12671, was published on 18 February 2025. ^[3][1] The model is a medical-focused entry in the company's line, and later medical models from the company are described in separate entries. The benchmark figures are Baichuan's own, so readers should check the report for the test conditions.