ModelBest releases MiniCPM4 for end devices

Background

A language model that has to run on an end-side device — a phone, a laptop, a car's computer — works under constraints that do not apply in a data center. Memory bandwidth, battery capacity and heat dissipation limit how much of the model can be read per second, and a user expects an answer without waiting for a round trip to a server. Three techniques come up again and again in this setting. Sparse attention lets a model look at a small, selected part of a long context instead of every earlier token. Quantization stores weights and activations at lower precision, so fewer bits move through memory. Speculative sampling uses a cheap draft to propose several tokens that a larger model then checks in a single pass. Each technique trades a little quality for a large saving in compute or memory.

What was released

On 9 June 2025 the MiniCPM team published MiniCPM4 together with its technical report, titled MiniCPM4: Ultra-Efficient LLMs on End Devices ^[1]. The model is built for end-side devices rather than for server deployment ^[1]. Open weights appeared on Hugging Face under the Apache-2.0 license, and the model card for MiniCPM4-8B was created on 5 June 2025, a few days before the report came out ^[2]. Apache-2.0 is a permissive license: it allows commercial use, modification and redistribution as long as the notice is kept, which matters for anyone who wants to ship a model inside a product.

Training data

The team credits two datasets, UltraClean and UltraChat v2, with carrying the model to satisfactory performance over 8 trillion training tokens ^[1]. The scale is the interesting part. Training on trillions of tokens is expensive, and a large share of that cost can be wasted on duplicated, noisy or badly filtered text. Treating data cleaning as a first-class part of the recipe, rather than as a preprocessing detail, is one of the ways a smaller model is asked to make more of what it reads.

The inference stack

The release introduced InfLLM v2, a sparse attention mechanism ^[1]. It also introduced BitCPM, a ternary quantization method ^[1]. Ternary quantization reduces weights to three levels, which sharply cuts both the memory needed to hold a model and the bandwidth needed to read it. Beyond the individual components, the team describes an inference system that combines sparse attention, quantization and speculative sampling in one stack ^[1]. The three techniques attack different bottlenecks — the cost of attention over long contexts, the memory footprint of the weights, and the serial, one-token-at-a-time nature of generation — so fitting them together is what decides whether a model actually runs at a usable speed on a small device.

Why it matters

Because the weights are published under Apache-2.0 ^[2], other groups can inspect the model, re-run the evaluations, fine-tune it for their own languages or tasks, and ship it in products. The components also stand on their own: sparse attention, ternary quantization and speculative sampling are described in the report and can be adopted by other projects ^[1]. For the wider question of where language models run, MiniCPM4 is a data point in the argument that a capable model does not have to live in a data center, and its training and inference choices are the parts that other teams can examine and reuse.