SAM 2 (Segment Anything Model 2)

What it is

SAM 2 is a unified model for real-time promptable object segmentation in images and videos. Meta released it on 29 July 2024 as an extension of the original SAM from still images to video.^[1] For single images, its memory is empty and it behaves like SAM, so one model covers both cases.^[1]

Origins

SAM 2 was released with a new dataset, SA-V, containing about 51,000 videos and more than 600,000 masklets. Those videos were collected across 47 countries.^[1] A masklet is a mask tracked through a series of video frames. The licences were split by part: code and weights under Apache 2.0, SA-V under CC BY 4.0, and the evaluation code under BSD-3.^[1]

Main features

Prompts. A prompt can be a positive or negative click, a box or a mask. In video, the model propagates a mask across frames into a masklet, which can be refined on any frame.^[1]

Streaming design. SAM 2 processes video one frame at a time. It uses a memory encoder, a memory bank that holds past frames and prompted frames, and memory attention, which lets the current frame draw on that stored history. An occlusion head predicts whether the object is visible, which matters when another object blocks it.^[1]

Speed. The model runs at roughly 44 frames per second.^[1]

Interactive video. Across 17 zero-shot datasets for interactive video segmentation, SAM 2 needed about three times fewer human interactions than prior approaches.^[1]

Images. On images, SAM 2 is more accurate than the original SAM and six times faster across a 23-dataset zero-shot suite.^[1]

Annotation. Using SAM 2 in the annotation loop was 8.4 times faster than annotating each frame with the original SAM.^[1]

Why video is harder. Following one object through a video is harder than segmenting it in a single image. The object can change shape, be partly hidden by another object, or leave the frame and come back. A model that carries a mask forward has to handle all of this, which is what the memory design is for. Take a user who clicks a car in the first frame of a clip. The model turns that click into a mask and carries it forward. If the car passes behind a tree, the occlusion head signals that it is hidden, and the user can add a corrective click on a later frame instead of starting again.^[1]

Its place in development

SAM 2.1 came out in the autumn of 2024. It was an improved checkpoint with better handling of visually similar objects and of occlusions. On 12 February 2025 it became available on Amazon SageMaker JumpStart.^[1]

SAM 3 followed on 19 November 2025. Its tracker builds on the memory bank and memory encoder from SAM 2, so SAM 2's design carried into the next generation.^[2][3]