MiniMax releases natively multimodal MiniMax-M3 with sparse attention

Background

M3 was the next model in MiniMax's series. The release combined two changes: a model trained on several kinds of input from the start, and a new attention design for long inputs.

What happened

MiniMax released M3 in June 2026. Its weights were posted to Hugging Face on 2 June 2026, and the paper, arXiv 2606.13392, followed on 11 June ^[2][3][1]. Hugging Face is a platform where researchers and developers share models and datasets.

Key details

M3 is a mixture-of-experts model with about 428 billion total parameters, about 23 billion of which are activated ^[1]. It supports a 1-million-token context ^[1], and it is natively multimodal, trained on text, images and video from the start ^[1].

The release introduced MiniMax Sparse Attention (MSA), which MiniMax designed to replace GQA-style attention for long contexts. Attention is the mechanism that lets a model weigh earlier parts of its input when producing each new token, and grouped-query attention (GQA) is a common way of making it cheaper ^[1]. Because attention touches every part of processing that depends on input length, changing it affects a large share of the work the model does with long inputs.

At the full 1-million-token context, MiniMax reported 9 times faster prefill and 15 times faster decoding than M2. Prefill is the stage in which the model reads its input and decoding is the stage in which it generates output. Per-token compute was cut to one-twentieth ^[1].

Significance

The main claims of the release concern long-context efficiency and the combination of text, image and video in one model. Both are set out in the figures above, and the paper is the primary reference for them ^[1]. Long-context efficiency matters because the cost of processing a very long input grows quickly with its length, so a method that reduces that cost widens what a model can practically read in one pass.