Jais-3-MoVA-36B-A4B

What it is

Jais-3-MoVA-36B-A4B is a language model from the Institute of Foundation Models (IFM) built with Arabic at its centre. The numbers in its name describe how it is put together: it stores 36 billion parameters in total, but only about 4 billion of them are used for any single token it generates^[1].

That gap is the point. In a conventional "dense" model every parameter takes part in every computation, so cost grows with the total size of the model. A mixture-of-experts (MoE) model instead splits its feed-forward layers into many separate blocks, or experts, and a routing step sends each token to only a few of them. The model keeps a large store of parameters while paying, per token, roughly what a much smaller model would cost. Most of Jais-3's 36 billion parameters sit idle at any one moment.

The MoVA part of the name

MoVA stands for mixture of values. In most mixture-of-experts models the saving happens only in the feed-forward layers, while the attention layers stay dense and every token passes through all of their parameters. MoVA applies the same sparse, routing-based idea on the attention side, layered on top of the mixture-of-experts feed-forward layers^[1]. The name refers to the value vectors inside attention — the part of the mechanism that carries information between positions in a sequence — which are mixed selectively rather than used in full.

One further architectural detail is unusual: the context window is 512,000 tokens^[1]. That is long enough to hold whole books or large codebases in a single prompt, which matters for agent-style work where a model accumulates tool output over many steps.

Footprint

A 36-billion-parameter model is small enough to be practical. IFM states that it fits on a single H200 at about 75 GB in BF16^[2]. In this design the serving cost follows the active parameter count rather than the stored one, so the memory and bandwidth needed per token are far below what a dense model of the same total size would demand.

Quantised builds — FP8, GGUF and MLX — are published under a separate IFM organisation on Hugging Face^[1]. The three cover different deployment routes: FP8 is a low-precision format for data-centre accelerators, GGUF is the format used by local inference tools, and MLX is Apple's framework for running models on its own chips.

Benchmarks

IFM reports an Arabic average of 82.5 for this model, within a point of the 83.1 scored by a dense 32-billion-parameter model^[2]. On the agentic average it scores 57.4, against 60.0 for the dense model^[2]. The exception is tool use: on MCPMark it reaches 69.3, ahead of the 32-billion-parameter model at 66.1^[2].

So the trade is not a straight win or loss. Against a dense model of comparable size, Jais-3-MoVA gives up a little on general Arabic ability and on the agentic average, gains on the tool-use benchmark, and activates a small fraction of its parameters for each token.

Where it fits

IFM announced the model on 15 October 2026 as part of a six-model family^[2], and positions this size for fast, low-cost agents running at scale^[2]. Sparse activation, a 512,000-token context, a footprint that fits on one accelerator, and published quantised builds all point at the same case: an agent that is called many times in parallel, where cost and latency per call dominate the bill. The competitive Arabic scores suggest the sparse design is not paying for itself with a large loss in quality, and the tool-use result suggests that long-context agentic workloads were part of the target from the start.