MiMo-7B

What MiMo-7B is

MiMo-7B is a language model with 7 billion parameters, built by Xiaomi's LLM-Core team for reasoning tasks and optimized at both the pre-training and post-training stages^[1]. Two variants are central to the release: MiMo-7B-Base, the pre-trained base model, and MiMo-7B-RL, the version subsequently tuned with reinforcement learning^[1]. The Hugging Face model card for MiMo-7B-RL was created on 29 April 2025, and the model is published under the MIT license^[3].

Background

The technical report from Xiaomi's LLM-Core team was first submitted on 12 May 2025^[1]. Its subject is the reasoning ability of a language model across the whole pipeline, from pre-training to post-training, and the design choices below follow that split: first give the base model as much reasoning potential as possible, then use reinforcement learning to draw it out^[1].

Pre-training

The base model was pre-trained on 25 trillion tokens^[1]. To strengthen reasoning potential, the team used a three-stage data-mixing strategy, changing the composition of the training data across stages rather than holding one mixture fixed throughout^[1]. The model also adds a multi-token prediction objective: in addition to predicting the next token, it is trained to predict several upcoming tokens at once. That makes the training signal denser at each position and, according to the report, also speeds up inference^[1].

Post-training

Post-training centred on reinforcement learning over verifiable problems: 130K mathematics and programming problems whose answers can be checked automatically^[1]. In this setup the model generates a solution, a checker decides whether it is correct, and that verdict becomes the reward — a way to train reasoning without human preference labels. Two adjustments address the practical difficulties of the approach. A test-difficulty-driven code reward was introduced to alleviate sparse rewards: in code tasks a single failing test case can zero out an otherwise reasonable attempt, so the reward is shaped by how hard the tests themselves are. Data resampling was used to stabilise training^[1].

Reported results

The team reports that MiMo-7B-Base outperforms much larger 32B models^[1]. After reinforcement learning, MiMo-7B-RL is reported to surpass OpenAI o1-mini on mathematics, code and general reasoning^[1].

The 30 May 2025 update

An update on 30 May 2025 scaled the supervised fine-tuning dataset from about 500K to 6M instances and widened the reinforcement learning window from 32K to 48K^[2]. With these changes, AIME 2024 rose from 68.2 to 80.1, above the 79.8 reported for DeepSeek R1^[2]. AIME 2024 refers to the 2024 edition of the American Invitational Mathematics Examination, a competition whose problems are widely used as a benchmark for mathematical reasoning.

Where it sits

MiMo-7B belongs to the group of relatively small models built specifically for reasoning tasks^[1]. The comparisons the team makes are against larger models and against OpenAI o1-mini^[1], and the MIT license means the weights can be downloaded and reused^[3]. The later update is a reminder of where the gains came from: the scale of the fine-tuning data and the length of the reinforcement learning window remained direct levers on benchmark scores^[2].