Segment Anything Model (SAM)

What it is

Segmentation means deciding which pixels of an image belong to which object, producing a mask that traces each object's outline. The Segment Anything Model, usually called SAM, is a segmentation model that is promptable: the user points at or outlines what they want, and the model returns the mask for that object. Meta AI released it on 5 April 2023 and presented it as a step toward a foundation model for image segmentation, meaning one general model that can serve many segmentation tasks.^[1]

Origins

The paper, *Segment Anything*, was posted to arXiv under the identifier arXiv:2304.02643.^[2] Meta released the model together with SA-1B, a dataset of image masks built by a three-stage data engine. The engine moved from model-assisted labelling, to semi-automatic labelling, to fully automatic labelling.^[1]

Main features

Prompts. A prompt can be foreground or background points, a rough box or a mask, or other information. For an ambiguous prompt, the model returns a valid mask for one of the possible objects.^[1]

Two jobs in one model. The same model handles interactive segmentation, where a person guides it, and fully automatic segmentation of every object in an image.^[1]

Zero-shot transfer. The model transfers to new domains without extra training. Meta cited underwater photos and cell microscopy as examples.^[1] Zero-shot here means the model is used on kinds of images it was not specifically trained on.

Architecture. An image encoder computes an image embedding once. A lightweight prompt encoder turns any prompt into a form the model can use in real time, and a lightweight decoder predicts the masks.^[1] Because the heavy encoder runs only once per image, each new prompt is cheap to process.

Speed. Once the image embedding is computed, the model produces a mask for any prompt in about 50 milliseconds, and this can run in a web browser.^[1]

SA-1B dataset. SA-1B contains over 1.1 billion masks on about 11 million licensed, privacy-preserving images. Meta said this was 400 times more masks than any existing segmentation dataset, and it was released for research purposes.^[1]

Labelling speed. Assisted annotation took about 14 seconds per mask, which Meta said was 6.5 times faster than COCO polygon annotation.^[1]

Licence and access. The model was released under the permissive Apache 2.0 license, with code on GitHub and a demo at segment-anything.com.^[1]

Its place in development

Meta used Segment Anything in its Backdrop image-editing feature for Instagram, announced in September 2023.^[3] The approach was then extended to video with SAM 2, released in July 2024, which added a memory mechanism to follow objects across frames.^[4] SAM 3, released in November 2025, added text and exemplar prompts.^[4][5]

The first SAM showed that one prompt-driven model could work across very different kinds of images, and that a large, carefully labelled dataset could be built with the model's help.^[1] Its successors kept the same basic idea of prompting a segmentation model, with each generation adding new kinds of input.^[4][5]