Phi-4

What Phi-4 is

Phi-4 is a 14-billion-parameter dense decoder-only Transformer released by Microsoft on 12 December 2024 and described in the Phi-4 Technical Report^[1][2]. "Dense" means every parameter takes part in the computation for each token, unlike mixture-of-experts models that route a token through only part of the network. "Decoder-only" means the model is a stack of self-attention and feed-forward blocks that produces text one token at a time. It accepts contexts of up to 16K tokens^[2]. At 14 billion parameters it sits in the small-to-mid range — the kind of model that can be served on modest hardware rather than a large cluster.

Where it came from

Phi-4 is the successor of Phi-3.5 in Microsoft's Phi family of small language models^[1]. The recipe behind it is centred on data quality: synthetic data is used throughout pre-training, mid-training and post-training, seeded from curated web content, licensed books and code^[1]. In practice that means a comparatively small set of hand-picked sources is used to generate training material, and the generated material appears at every stage of training rather than only at the end.

The training run

Training used 9.8 trillion tokens on 1,920 H100-80G GPUs for 21 days, between October and November 2024; the public data cutoff is June 2024^[2]. The context window is 16K tokens^[2].

Results

The technical report says Phi-4 significantly exceeds its teacher model, GPT-4o, on GPQA and MATH^[1]. GPQA is a set of graduate-level science questions and MATH is competition-style mathematics, so both probe multi-step reasoning rather than recall. The model card lists MMLU 84.8, GPQA 56.1, MATH 80.4, HumanEval 82.6 and MGSM 80.6; the SimpleQA score is 3.0, which the card ties to limited factual knowledge^[2].

Benchmark scores only mean something if the test items were not seen during training, so the team also checked for contamination. Phi-4 was evaluated on the November 2024 AMC-10 and AMC-12 competitions, which were held after its training data had been collected; The Decoder reports a score of 91.8 on them^[1][3].

Release and position

Phi-4 was initially offered on Azure AI Foundry; on 8 January 2025 Microsoft released the full weights on Hugging Face under the MIT licence^[2][3]. Publishing weights under a permissive licence means outside developers can download and run the model themselves, not just call it through a hosted service. Microsoft positioned Phi-4 for memory- and compute-constrained environments, latency-bound scenarios and reasoning and logic work, primarily in English^[2].

It is the basis for Phi-4-reasoning (April 2025) and Phi-4-reasoning-vision-15B (March 2026)^[1].

Limitations

The model card warns that Phi-4 was trained mainly on English text and is not intended for multilingual use, and that it performs worse on other languages^[2]. It also notes that most of the training data is Python-based, so scripts using other packages or languages should be verified manually^[2]. The low SimpleQA score points the same way: when a task depends on recalling specific facts, the output needs checking rather than trusting^[2].