Focal Loss for Dense Object Detection

The problem

Focal Loss for Dense Object Detection was first submitted to arXiv on 7 August 2017 as 1708.02002. Its authors are Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollar.^[1] The paper begins with a contrast in object detection. At the time, the most accurate detectors were two-stage approaches in the R-CNN style: they apply a classifier to a sparse set of candidate locations.^[1] A one-stage detector instead works over a dense sampling of locations. In principle this should make it faster and simpler, because it avoids a separate sparse candidate stage. In practice, however, one-stage detectors had trailed two-stage detectors in accuracy.^[1]

Diagnosis

The authors identify the extreme foreground-background class imbalance in dense detectors as the central cause.^[1] Dense sampling means the detector evaluates a very large number of locations; most are background, and only a small fraction correspond to objects. If the training objective gives every location a similar role, the many background examples can dominate the total loss. Even when each easy background example contributes only a small loss, their combined contribution can be large. The model then spends much of its training effort on examples it already classifies well, rather than on the harder cases that matter for detection quality.

Method

The paper proposes reshaping the standard cross entropy loss so that it down-weights the loss assigned to well-classified examples; the authors call the resulting loss the focal loss.^[1] In ordinary cross entropy, a well-classified example still has a small, non-zero loss. When such examples are extremely numerous, their total effect can still be substantial. Focal loss reduces the weight of those easy examples, so the training signal concentrates more on examples the model has not yet classified well.^[1] The proposal is a change at the level of the loss function. It does not introduce a new two-stage candidate pipeline; it changes how the dense samples are weighted during training.

Why it matters

The significance of the paper is that it names a concrete obstacle to dense detection and offers a direct response. The obstacle is imbalance: dense sampling creates far more background locations than object locations.^[1] The response is to alter the loss so that easy, well-classified examples do not dominate.^[1] This reframes the accuracy gap between one-stage and two-stage detectors, at least in part, as a problem of training objective rather than only architecture.^[1]

For a reader learning detection, the lesson is that detector quality is not only a matter of adding stages or features. The distribution of training examples and the way their losses are weighted also shape what the model learns. The paper makes that point by taking a dense detector's overwhelming background population seriously and turning it into a design target for the loss.^[1]