Llama 4 Behemoth

What it is

Llama 4 Behemoth is a very large mixture-of-experts model from Meta. Meta previewed it on 5 April 2025 alongside Llama 4 Scout and Maverick, but said it was still training and would not be released yet.^[1] It has 288 billion active parameters, 16 experts and nearly 2 trillion total parameters.^[1]

Behemoth's role was as a teacher. Llama 4 Maverick was codistilled from it, which means Maverick learned from Behemoth's outputs during training.^[1] A teacher model is a larger model whose behaviour a smaller model is trained to copy.

Origins and training

Meta pretrained Behemoth with FP8 precision on 32,000 GPUs, reaching 390 TFLOPs per GPU. FP8 is a lower-precision number format, which uses less memory and less time for each calculation than higher-precision formats.^[1]

After pretraining, Meta pruned 95% of the supervised fine-tuning data for Behemoth. For the smaller Llama 4 models the figure was about 50%. Meta also said a revamped reinforcement-learning infrastructure gave about ten times the training efficiency.^[1] Supervised fine-tuning is training on examples of good answers, and reinforcement learning trains the model on rewards for better behaviour.

Main features

Meta's claims. Meta said Behemoth outperformed GPT-4.5, Claude Sonnet 3.7 and Gemini 2.0 Pro on STEM benchmarks such as MATH-500 and GPQA Diamond.^[1] A benchmark is a standard test set, and these results are Meta's own reports rather than independent tests.

Scale. Behemoth's total size is nearly 2 trillion parameters, far larger than any other Llama 4 model. Only 288 billion of those parameters are active at once, because the model uses the mixture-of-experts design.^[1] Models of this size need very large clusters of GPUs to train, as the 32,000 GPUs used for its pretraining show.^[1]

Its place in development

Behemoth's release was delayed. In May 2025 the Wall Street Journal reported that its release, originally targeted for April, had slipped to autumn or later, amid concerns about limited gains over earlier models. Meta declined to comment.^[2]

On 14 July 2025 the New York Times reported that leaders of the new Meta Superintelligence Labs, including Alexandr Wang, had discussed abandoning Behemoth in favour of developing a closed model. Meta declined to say whether the model had been discontinued.^[3] A closed model is one whose weights are not published, so only its owner or partners can run it.

Behemoth was never publicly released. As of January 2026, Llama 4 remained Meta's most recent Llama release.^[4] Meta's next frontier effort came from Meta Superintelligence Labs, and its first model was Muse Spark, released on 8 April 2026.^[5]

For a learner, Behemoth shows that a model can matter even when it is never released. Its role as a teacher shaped Maverick, and the reports about its delay and possible replacement are the main public record of what happened to it.^[1][2][3]