DeepSeek-VL2

What DeepSeek-VL2 is

DeepSeek-VL2 is a family of mixture-of-experts vision-language models released by DeepSeek, with an arXiv report dated 13 December 2024^[1][3]. It follows DeepSeek-VL, the lab's earlier vision-language model, and is positioned as its successor^[1]. Rather than a single model, the release covers three sizes: DeepSeek-VL2-Tiny with 1.0B activated parameters, DeepSeek-VL2-Small with 2.8B, and DeepSeek-VL2 with 4.5B^[1]. The sizes are given in activated parameters — the slice of the network that actually runs for each token of input. A mixture-of-experts model holds more parameters in total and routes each token through a subset of them, so the number in the name is smaller than the full parameter count.

How the two halves work

A vision-language model pairs an image path with a language model, and the design questions are how to feed in a picture and how to keep inference affordable.

On the vision side, DeepSeek-VL2 uses a dynamic tiling strategy to handle high-resolution images with different aspect ratios^[1]. Instead of resizing every picture to one fixed square, which loses fine detail in dense documents and distorts shapes, the image is cut into tiles whose number and layout adapt to its shape. Small text on a scanned page or the tick labels of a chart therefore reach the language model at higher effective resolution than a single downscaled copy would carry.

On the language side, the model uses DeepSeekMoE architectures with Multi-head Latent Attention^[1]. Multi-head Latent Attention compresses the key-value cache, the stored intermediate state a transformer keeps for the tokens it has already read, so that inference costs less memory^[1]. Pairing sparse experts with a compressed cache is what allows the family to keep activated parameter counts in the low single-digit billions while still running a large language backbone.

What it is built to do

The stated targets are visual question answering, OCR, and document, table and chart understanding, plus visual grounding^[1]. These are tasks where a multimodal model has to read text off an image, keep the layout of a page or a table straight, and, for grounding, indicate where in the image a described object is. DeepSeek reports competitive or state-of-the-art results against open dense and mixture-of-experts models with similar or fewer activated parameters^[1].

Release, licence and tooling

The repository is released under the MIT License, while use of the models themselves is governed by the DeepSeek Model License, which supports commercial use^[2]. Tooling followed the weights: on 25 December 2024 the release notes listed a Gradio demo example, incremental prefilling and support for VLMEvalKit^[2]. A naive Gradio demo for deepseek-vl2-small was published on Hugging Face Spaces on 6 February 2025^[2]. Incremental prefilling is the kind of feature that matters when a session is extended step by step rather than processed once from the beginning.

Where it sits

DeepSeek-VL2 continues the line begun by DeepSeek-VL, and its place in the field is defined by the trade it makes: mixture-of-experts language backbones with compressed attention caches, paired with tiling for high-resolution input, at activated-parameter counts from 1.0B to 4.5B^[1]. Wikipedia records the release date as 13 December 2024^[3].