Meta releases Llama 3.3 70B

Background

By late 2024 the Llama 3 line already had large models, including a 405B version from Llama 3.1. Large models are expensive to run, so a common question for developers was whether a smaller model could handle the same everyday tasks. Llama 3.3 was Meta's answer for text tasks at the 70B size.^[2] The number in a model's name, such as 70B, is its parameter count in billions. More parameters generally means a larger model that needs more memory and computing power to run.

What happened

On 6 December 2024 Meta released Llama 3.3, a 70B text-only instruction-tuned model.^[1][2] Instruction tuning is extra training that teaches a base model to follow user requests in a chat format. "Text-only" means the model reads and writes text and does not take images as input.

Key details

Comparison with 405B. Meta said Llama 3.3 performs similarly to Llama 3.1 405B at much lower cost.^[2] Lower cost here means it takes far less hardware and energy to run each request.

Benchmarks. On IFEval, a test of following detailed instructions, Llama 3.3 scored 92.1. On MATH, a test of mathematical problem solving, it scored 77.0. Llama 3.1 405B scored 88.6 and 73.8 on the same two tests. So the 70B model scored higher on both.^[1] A benchmark score is a single measure on a fixed test set, so it shows relative performance on those tests rather than on every task.

Context and languages. The model has a 128K-token context and supports eight languages.^[1] A token is a small piece of text, roughly a word or part of one, and the context is how much text the model can take in at once.

Licence. It was released under the Llama 3.3 Community License Agreement.^[1] A community licence sets terms for use, which Meta's own agreement defines for Llama models.

Place in the series. Llama 3.3 was the last Llama 3 release before Llama 4.^[2]

Impact

Llama 3.3 showed, on the benchmarks Meta cited, that a 70B model could perform similarly to the older 405B model. That gave developers a smaller and cheaper option for many text applications. Its licence allowed organisations to use the model under the community terms. As the final model in the Llama 3 line, it closed one stage of the series before Llama 4 arrived.^[1][2]