Mask R-CNN

Overview

Mask R-CNN is a framework for object instance segmentation. Given an image, it detects objects and produces a high-quality segmentation mask for each instance ^[1]. Object detection typically returns a bounding box and a category for each object; instance segmentation goes further by assigning pixels to each individual object. Mask R-CNN handles both jobs in one model.

The paper was first submitted to arXiv on 20 March 2017 as 1703.06870 ^[1]. Its authors are Kaiming He, Georgia Gkioxari, Piotr Dollar and Ross Girshick ^[1].

How it works

The method extends Faster R-CNN. Faster R-CNN already has a branch for bounding box recognition. Mask R-CNN adds a second branch that predicts an object mask in parallel with that existing branch ^[1]. In other words, the model does not replace detection with segmentation; it keeps the detector and asks an extra branch to output a mask for each detected instance.

The authors describe the framework as simple to train ^[1]. They also report that it adds only a small overhead to Faster R-CNN and runs at 5 frames per second ^[1]. Those two properties matter for a model that outputs masks: mask prediction adds work beyond box prediction, but here the extra cost is described as small.

Results

The paper reports top results in all three tracks of the COCO suite of challenges ^[1]. The authors also show that the framework generalises to other tasks. One example is estimating human poses within the same model ^[1]. This goes beyond producing masks for objects: the same structure can be used for a different output that also has a spatial layout.

Why it matters

Mask R-CNN's significance comes from combining two outputs that are often handled separately: object detection and per-instance segmentation. The detector supplies bounding boxes and recognition, while the added branch supplies masks ^[1]. Because the mask branch runs in parallel, the design keeps the detector's structure and adds a relatively small amount of computation ^[1].

The model is also presented as a general framework rather than a single-purpose segmentation system. Its ability to estimate human poses in the same model is evidence for that generality ^[1]. In practical terms, a system that can detect objects, segment them, and support related spatial tasks in one model is easier to describe and train than a collection of unrelated pipelines; the authors specifically describe Mask R-CNN as simple to train ^[1].

Scope

The paper's claims are bounded by what it reports: a mask-prediction branch added to Faster R-CNN, top results across the three COCO tracks, 5 frames per second, and generality to human pose estimation ^[1]. It does not present instance segmentation as an isolated task; it presents it as an extension of an object detector. That framing is the core of the paper: detection and mask prediction share one framework, and the mask branch operates in parallel with the box branch ^[1].