Mixtral 8x22B

What it is

Mixtral 8x22B is a sparse mixture-of-experts (MoE) language model from Mistral AI. It has 141 billion parameters in total, but only 39 billion of them are used for any single token ^[1]. Its context window is 64K tokens ^[1], and Mistral lists it as fluent in English, French, Italian, German and Spanish, with native function calling ^[1].

"Mixture of experts" describes a way of arranging a network so that only part of it runs for a given input. The feed-forward portion is split into a set of separate sub-networks, usually called experts, and a router decides which few of them handle each token. Because most experts stay idle for any one token, the model can store far more parameters than it spends computing a single token. That is the sense in which the design is sparse: large in total size, partial in use.

Where it came from

Mistral AI announced the model on 17 April 2024 ^[1], while Wikipedia dates the release of its weights to 10 April 2024 ^[2]. It is the successor to Mixtral 8x7B ^[4]. The weights were published under the Apache 2.0 licence, which Mistral described as the most permissive open-source licence ^[1].

What it was built for

Mistral describes Mixtral 8x22B as optimized for reasoning ^[1]. On mathematics benchmarks, the instructed version scored 90.8% on GSM8K with maj@8 and 44.6% on MATH with maj@4 ^[1]. GSM8K is a collection of grade-school math word problems; MATH is a harder set of competition-style problems. The maj@8 and maj@4 notation means the model was sampled several times and the most frequent answer was taken, rather than relying on a single attempt.

The company also says the model strongly outperforms LLaMA 2 70B on HellaSwag, Arc Challenge and MMLU ^[1]. HellaSwag and Arc Challenge are multiple-choice tests of everyday and scientific reasoning, while MMLU covers knowledge across many academic subjects.

On speed, Mistral says the model's sparse activation patterns make it faster than any dense 70B model ^[1]. A dense model of that size runs all of its parameters for every token; Mixtral 8x22B activates 39 billion out of its 141 billion, so the compute spent on each token is lower than the stored parameter count would suggest.

On the API, and retirement

From 17 April 2024 the model was served on Mistral's API under the name open-mixtral-8x22b ^[3]. Mistral deprecated it on 30 November 2024 and retired it on 30 March 2025 ^[4]. Its model overview lists Mistral Small 4 as the alternative to the retired model ^[4].

Why it mattered

Mixtral 8x22B sat at the large end of open-weight releases: 141 billion parameters, published under a licence that permits commercial use and modification ^[1]. Its sparse design also served as an argument that a model can carry a very large parameter count without paying for all of it on every token, since only 39 billion parameters are active per token ^[1]. In Mistral's own line-up it followed Mixtral 8x7B ^[4] and was in turn replaced by Mistral Small 4 ^[4]. Announced in April 2024 and retired in March 2025, it was available for less than a year, a short life that says as much about how quickly the field moved as about the model itself ^[1][3][4].