DeepSeek releases the DeepSeek-VL2 vision-language models

Background

A vision-language model combines image understanding with text generation, so that it can answer questions about pictures, read documents and interpret charts. DeepSeek had earlier released the vision-language model DeepSeek-VL, and the new series is its follow-up.

For vision-language models, two problems stand out. Images come in many shapes and resolutions, and text inside a document needs to be read at a fine level of detail. Mixture-of-experts designs are one way to keep the computation per token low while the model stays large.

What happened

On 13 December 2024 DeepSeek released the DeepSeek-VL2 series, the date of its arXiv report ^[1][3]. The series has three variants: DeepSeek-VL2-Tiny with 1.0B activated parameters, DeepSeek-VL2-Small with 2.8B, and DeepSeek-VL2 with 4.5B ^[1].

The series targets visual question answering, OCR, document, table and chart understanding, and visual grounding ^[1]. Visual grounding means finding the part of an image that a phrase refers to.

Key details

The vision side uses a dynamic tiling strategy to handle high-resolution images with different aspect ratios ^[1]. Tiling splits an image into pieces that a fixed-size input can process, and a dynamic approach chooses the split according to the image's shape. The language side uses DeepSeekMoE models with Multi-head Latent Attention, which compresses the key-value cache for efficient inference ^[1].

DeepSeek reported competitive or state-of-the-art results against open dense and mixture-of-experts models with similar or fewer activated parameters ^[1]. These are the company's own comparisons.

Impact

The release gave developers and researchers a series of vision-language models in several sizes, so a user can choose a smaller variant for lower cost or a larger one for stronger results. The series also applies the mixture-of-experts design to images, with DeepSeekMoE on the language side ^[1]. The event is a useful reference for anyone studying how the same efficiency ideas are applied to multimodal models.