Llama 3.3 70B

What it is

Llama 3.3 70B is a text-only language model with 70 billion parameters, instruction-tuned to follow user requests. Meta released it on 6 December 2024, and it was the last Llama 3 release.^[1][2] The model supports a 128K-token context and eight languages, and it was pretrained on over 15 trillion tokens with a December 2023 knowledge cutoff.^[1]

Instruction tuning is a further training step in which the model learns from examples of requests and good responses, so it behaves like an assistant rather than only continuing text. The base model learns only to predict text. The instruction-tuned version is the one used in chat.

Origins

Ahmad Al-Dahle, Meta's VP of generative AI, announced the model on X. Mark Zuckerberg shared the news on Instagram.^[2] Training used 7.0 million H100 GPU hours.^[1]

Meta's emphasis was on efficiency. It said the model delivers performance similar to Llama 3.1 405B at much lower cost, and that it can run locally on a typical developer workstation.^[2] The model was released under the Llama 3.3 Community License Agreement.^[1]

Main features

Comparison with the 405B model. Meta compared Llama 3.3 70B with Llama 3.1 405B on several benchmarks. Scores for the 70B model against the 405B model were 86.0 against 88.6 on MMLU (CoT), 68.9 against 73.3 on MMLU Pro, and 92.1 against 88.6 on IFEval. They were 50.5 against 49.0 on GPQA Diamond, 88.4 against 89.0 on HumanEval, 77.0 against 73.8 on MATH and 91.1 against 91.6 on MGSM.^[1] The 70B model is ahead on some tests and slightly behind on others, which is why the matches are described as similar rather than identical.

How to read the scores. Each benchmark named here tests a different skill. MMLU covers knowledge across many academic and professional subjects. IFEval checks whether a model follows explicit formatting and content instructions. MATH uses competition-style math problems, GSM8K uses grade-school word problems, and HumanEval asks the model to write code that passes tests. Looking at several scores together gives a fuller picture than any single number.

Improvement over Llama 3.1 70B. Compared with the earlier 70B model, Llama 3.3 improved MATH from 68.0 to 77.0, HumanEval from 80.5 to 88.4 and IFEval from 87.5 to 92.1.^[1]

Its place in development

The model sits at the end of the Llama 3 line, and the Llama 4 herd followed it.^[1][4]

At LlamaCon, held on 29 April 2025, Meta's Llama API preview offered fine-tuning tools for a Llama 3.3 8B model.^[3] Fine-tuning means further training a model on a narrower set of examples so that it suits a specific job.

The next major step was the Llama 4 herd, announced on 5 April 2025. It was Meta's first set of mixture-of-experts Llama models.^[4]