Infinigence AI releases the Megrez-3B device-side model family

Background

Running a language model on a phone or a laptop means working within limits on memory and power, and without a round trip to a data centre. Infinigence AI designed the Megrez-3B family for that setting.

What was released

In December 2024, Infinigence AI released the Megrez-3B family in two variants, Instruct and Omni ^[1][2]. The company's comparison table dates both variants to 16 December 2024, while the Instruct weights were uploaded to Hugging Face on 15 December 2024 ^[1][2]. Both models are aimed at phones and other end-side hardware rather than data-centre serving ^[1].

The two models

Megrez-3B-Instruct is the text model. It has 2.92 billion parameters and a 32K context window ^[1]. The context window is the amount of text the model can take in at once, so a larger window lets it work over longer passages or conversations.

Megrez-3B-Omni adds image and audio input on top of a 2.29 billion parameter backbone, for about 4 billion parameters in total ^[2]. The company describes it as the first open-source omni-modal understanding model for devices ^[2]. Omni-modal means the model accepts more than one kind of input; the word understanding signals that it is built to interpret inputs rather than to generate images or speech.

Benchmarks and a demo

On the company's numbers, the Omni model averages 66.2 on OpenCompass multimodal benchmarks, above LLaVA-NeXT-Yi-34B ^[2]. These are figures reported by the company rather than independent measurements. Alongside the text model, Infinigence released InfiniWebSearch as a working web-search demo ^[3].

Open weights and where it sits

Both sets of weights are released under Apache-2.0 ^[1][2], a permissive open-source licence. That means outside developers can download the models and adapt them, which matters for a family whose stated target is local hardware rather than a hosted service ^[1].

The two variants cover different jobs at similar size: a text model with a long context window for reading and conversation, and a multimodal model that also takes images and audio ^[1][2]. By aiming at phones and other end-side devices, the family addresses a deployment setting distinct from data-centre serving ^[1]. InfiniWebSearch shows the text model wired into an application rather than presented only as weights ^[3].