Microsoft launches Phi-3-mini

Background

Phi-3 continued Microsoft's Phi line with a family of small language models, designed to be small enough to run locally.^[1][2]

What happened

On 23 April 2024 Microsoft introduced the Phi-3 family and launched Phi-3-mini first.^[1] Phi-3-mini has 3.8 billion parameters and comes in 4K and 128K context-length versions, which set how much text the model can take in at once.^[1] It was released on Azure AI Studio, Hugging Face and Ollama on the same day.^[1]

Microsoft said Phi-3-small (7B) and Phi-3-medium (14B) would follow. They arrived on Azure together with the 4.2-billion-parameter Phi-3-vision at Microsoft's Build conference on 21 May 2024.^[1][3]

Key details

The technical report, published on arXiv, says Phi-3-mini reaches 69% on MMLU, a multi-subject knowledge and reasoning test. It says this rivals Mixtral 8x7B and GPT-3.5, and that the model can run locally on a phone.^[2]

Phi-3-mini was trained on 3.3 trillion tokens of heavily filtered web data and synthetic data. When quantised to 4 bits, it takes about 1.8 GB of memory.^[2] Quantisation stores a model's numbers at lower precision, which shrinks its memory footprint at some cost in accuracy.

Impact

Phi-3 showed that a small model could be offered to developers through major cloud platforms, the open model hub Hugging Face and a local runner such as Ollama. That made the family a reference point for the idea that compact models can handle serious tasks on everyday hardware.^[1][2]

Terms used here

Ollama is a tool that lets people download and run language models on their own computer. Context length sets how much text a model can handle at once. Quantisation stores a model's numbers at lower precision so that the model fits in less memory. Parameters are the adjustable weights that training tunes, and the parameter count is a rough guide to a model's size.