Molmo

What Molmo is

Molmo is a family of open vision-language models released by Ai2 on 25 September 2024.^[1] A vision-language model takes an image together with a text prompt and answers in text: describing the scene, answering a question about it, or reading what is in it. Molmo drew attention for two reasons. On Ai2's own comparisons the models were competitive with well-known commercial systems, and the institute published not only the weights but the data behind them.^[1]

The family has four members of different sizes and roles. MolmoE-1B is a mixture-of-experts model with 1 billion active parameters out of 7 billion total, built on Ai2's OLMoE-1B-7B.^[1] Molmo-7B-O is described by Ai2 as the most open of the 7B models, Molmo-7B-D is the demo model, and Molmo-72B is the largest.^[1]

How it was built

All released models use OpenAI's CLIP ViT-L/14 at 336 pixels as the vision encoder, the component that turns pixels into representations the language side can work with.^[1]

Training used PixMo, a set of datasets built by Ai2 for this project.^[1] PixMo-Cap contains 712,000 images with about 1.3 million dense captions collected from speech descriptions.^[1] Collecting captions by speech is unusual: rather than typing a short line, people described the picture aloud, which Ai2 presents as a way of getting highly detailed image descriptions.^[2] PixMo-Points contains 2.3 million question-point pairs drawn from 428,000 images.^[1]

Pointing at things

The capability Ai2 emphasised most is pointing. Molmo can indicate specific objects in an image, not just describe the picture as a whole.^[1] Ai2 pitched pointing as the basis for several behaviours: counting objects one by one, explaining a scene in terms of the things in it, and guiding a robot or a web agent to a particular place.^[1] Pointing means the model gives a location in the image alongside its words; without it, a phrase such as "the cup on the left" stays ambiguous about which cup in the picture is meant.

Results Ai2 reported

Ai2 said Molmo-72B achieved the highest academic benchmark score in its comparison and ranked second in human preference, just behind GPT-4o, and it reported its best model outperforming Gemini 1.5 Pro and Flash and Claude 3.5 Sonnet.^[1] For the smaller models, Ai2 said MolmoE-1B nearly matched GPT-4V and that the 7B models fell between GPT-4V and GPT-4o.^[1] Ai2 also said its smaller Molmo models outperformed models ten times their size.^[2] These are Ai2's own comparisons, and the numbers are the institute's claims rather than independent measurements.

Openness

Weights, code, data and evaluations were released openly.^[1] Ai2 said its whole visual-language pipeline, comprising weights, code, data and evaluations, was open and free from distillation of proprietary vision-language models.^[2] Distillation here means training a model to imitate the outputs of a stronger model. Many open models are built partly that way, and Ai2's claim was that Molmo was not, so its abilities were meant to come from openly documented data instead.

Place in development

Molmo was succeeded by Molmo 2 in December 2025.^[1]