SmolLM3

What it is

SmolLM3 is an open language model from Hugging Face with 3 billion parameters, released on 8 July 2025 ^[1]. Parameters are the adjustable numbers inside a model that store what it has learned. SmolLM3 is the third generation of the SmolLM line, following SmolLM2 from November 2024 ^[1]. It is designed to be small enough for modest hardware while still handling long documents and reasoning tasks.

How it was built

The model was pretrained on about 11.2 trillion tokens in a three-stage curriculum ^[1]. A token is a small piece of text that a model reads during training. A curriculum means the training is split into stages, with the mix of data changing from one stage to the next. The model was then given post-training to shape it into a chat and reasoning model.

Long context was added in steps. The model was pretrained with a 4K-token window, then extended to 64K tokens in two stages of 50 billion tokens each. At inference time it can be used up to 128K tokens with YaRN, a method that stretches the position information so that the model can handle longer inputs ^[1].

Main features

SmolLM3 supports six languages: English, French, Spanish, German, Italian and Portuguese ^[1]. Its architecture is Llama-style, a decoder design that many open models share. It uses grouped-query attention with 4 groups, a way of arranging the attention layers to save memory ^[1]. It also uses NoPE, which removes rotary position embeddings from every fourth layer, a choice that helps performance on long contexts ^[1].

SmolLM3 is a dual-mode instruct model. Extended thinking is on by default, and users can switch it with the /think and /no_think flags ^[1]. With thinking on, its score on AIME 2025 rises from 9.3% to 36.7% ^[1]. AIME is a competition mathematics test, so this number shows how much the reasoning mode helps on hard problems.

The weights, both base and fine-tuned, are licensed under Apache 2.0 ^[2][3]. The license lets people use, change and share the models under its terms.

What Hugging Face claimed

Hugging Face said SmolLM3 outperforms Llama-3.2-3B and Qwen2.5-3B. It said the model is competitive with 4B models such as Qwen3-4B and Gemma3-4B ^[1]. These are the developer's own comparisons, and readers should check the evaluation setup behind them.

Openness

Hugging Face published the full recipe, the data mixtures, the training configurations, the logs and the intermediate checkpoints. The post-training data is in SmolTalk2 ^[1]. This level of disclosure lets other researchers see how the model was produced, reproduce parts of the process and build their own versions.

Where it fits

SmolLM3 continues the SmolLM2 line, with a larger token budget, longer context and a reasoning mode that can be turned on or off. It shows that a 3-billion-parameter open model can handle several languages, long documents and multi-step reasoning, and it gives people a small model whose training method they can inspect.