DeepSeek-V3

What it is

DeepSeek-V3 is an open-weight language model released by DeepSeek on 26 December 2024; the technical report was posted to arXiv the following day. ^[5][1]

It uses a mixture-of-experts (MoE) design. The model has 671 billion parameters in total, but only 37 billion of them are activated for each token it processes. A mixture-of-experts model divides its capacity among many specialist sub-networks and sends each token to a small number of them, so the amount of computation for each token is far smaller than the total parameter count suggests. The basic architecture stays within the Transformer framework and uses Multi-head Latent Attention (MLA) and DeepSeekMoE, both of which DeepSeek had first validated in DeepSeek-V2. ^[1]

Origins

DeepSeek-V3 was pre-trained on 14.8 trillion tokens, then refined with supervised fine-tuning and reinforcement learning. The context length was extended in two stages, first to 32K and then to 128K tokens. ^[1]

The report puts full training at 2.788 million H800 GPU hours. Pre-training accounted for 2.664 million of those hours and was completed in under two months on a cluster of 2,048 H800 GPUs. At an assumed rental price of $2 per GPU hour, that comes to $5.576 million. ^[1] The report stresses that this figure covers only the final training run and leaves out earlier research and ablation experiments. It also reports no irrecoverable loss spikes and no rollbacks during training. ^[1]

Main features

The report introduces an auxiliary-loss-free strategy for balancing load across experts, a multi-token prediction training objective, and FP8 mixed-precision training, which DeepSeek says it validated at this scale for the first time. ^[1]

On benchmarks for the chat model, DeepSeek-V3 scored 90.2 on MATH-500, 39.2 on AIME 2024 and 85.5 on Arena-Hard. DeepSeek describes it as the best-performing open-source model and says it is comparable to leading closed-source models. ^[1]

DeepSeek said the model generated 60 tokens per second, three times faster than DeepSeek-V2. ^[5]

The code is released under the MIT License. The weights use DeepSeek's Model License, which permits commercial use. Both the base and chat models are published on Hugging Face. ^[2][3]

From 8 February 2025, API prices were $0.27 per million input tokens on a cache miss, $0.07 per million on a cache hit, and $1.10 per million output tokens. ^[5]

Its place in development

DeepSeek-V3-Base is the starting point for the reasoning models DeepSeek-R1-Zero and DeepSeek-R1, released in January 2025. Later updates in the same line include DeepSeek-V3.1, released on 21 August 2025, and V3.2, released on 1 December 2025. ^[4]

The model also powered the DeepSeek mobile app, which launched on 15 January 2025. ^[6]