Infinigence AI publishes Megrez2 with cross-layer expert sharing

A preview and a finished model

On 22 July 2025, Infinigence AI published Megrez2-Preview on Hugging Face together with a technical report that set out the design behind it^[1][2]. The finished model, released as Megrez2-3x7B-A3B, followed on 12 September 2025^[1].

The problem: an impossible triangle

The report frames the target as device hardware — phones, laptops and other machines where the model runs locally — and describes the design problem as an impossible triangle of speed, accuracy and cost^[2]. A model that runs on a device has to fit into limited memory and storage, answer fast enough to be usable, and still be accurate enough to be worth running; improving one of the three usually costs one of the others.

Most large models are dense: every parameter is used for every token. A mixture-of-experts (MoE) model instead contains many separate expert sub-networks and routes each token through only a few of them, so the compute spent per token is much smaller than the total size of the model would suggest. The catch is that all the experts still have to be stored, including the ones a given token never touches.

Cross-layer expert sharing

Megrez2's first design choice attacks that storage problem. Under cross-layer expert sharing, three adjacent layers do not each keep their own experts; they draw on one shared pool of 64 experts^[2]. The result is a model that stores 7.5 billion parameters but activates 3 billion of them, which makes it smaller on disk than a 7B dense model^[2]. The name of the finished release, 3x7B-A3B, mirrors this: the stored and the activated parameter counts are different numbers, and the second is what roughly sets the compute cost per token.

Pre-gated routing

The second ingredient is pre-gated routing, which allows expert weights to be loaded before they are needed^[2]. Moving weights from storage into memory is slow compared with arithmetic, so a system that only begins fetching an expert after the router has chosen it spends part of every step waiting. Deciding early means the weights can be in place by the time the computation reaches them.

Training and reported numbers

Pre-training ran on 5 trillion tokens, with a final stage of more than 600 billion tokens used to reach a 32K context window^[2]. The release reports MMLU 85.4 and GPQA 58.8^[1], and the technical report puts C-EVAL at 91.7^[1][2].

Open weights

The weights are released under Apache-2.0, and the code lives in the Infini-Megrez repository^[1], so the model can be downloaded, inspected and run without a bespoke licence.

Where it sits

Models built for local hardware are usually judged by how much quality survives when the size budget shrinks. Megrez2's answer is to cut the number of stored parameters by sharing experts across layers, and to reduce the waiting that comes with fetching experts by deciding routes early — two changes aimed at the memory and latency sides of the same triangle the report starts from^[2]. The trade-offs of that arrangement depend on the workload, but the release leaves the design open to inspection: the report, the benchmark figures and the weights are all public^[1][2].