ILSVRC 2011 adds single-object localization

Background

The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) is a benchmark competition in large-scale object recognition. By the 2011 edition, the classification task had a familiar name at the top: XRCE, the team that had been runner-up in 2010, won the classification entry.^[1] This detail matters because it shows continuity at the top of the challenge: a group that had come close in the previous year returned and won.

Classification in 2011

XRCE's winning system combined high-dimensional image signatures with compression by product quantization and one-vs-all linear SVMs.^[1] An image signature is a vector that summarizes an image's appearance; high-dimensional signatures can be expensive to store and compare. Product quantization compresses such vectors by splitting them into smaller subvectors and encoding each subvector with a compact code, which reduces memory and computation. A one-vs-all linear SVM trains one linear classifier per class, asking for each class whether an image belongs to it or not. The combination is a classic pre-deep-learning pipeline: rich image representations, aggressive compression, and linear classifiers that can handle many classes.

A new localization task

The 2011 challenge also held the single-object localization competition for the first time, with two entries.^[1] In localization, a system must not only say what object is present but also indicate where it is; the task therefore tests the ability to propose and evaluate object regions. The UvA team won this first localization competition using a selective search approach to generate class-independent object hypothesis regions.^[1] Selective search does not start from a fixed class list; it combines simple image cues to propose regions that might contain objects, and those regions can then be classified. This class-independent design is important because the same region proposals can be shared across many object categories.

UvA's pipeline

The UvA entry followed region proposals with dense sampling and vector quantization of several color SIFT features, spatial pyramid matching, and a histogram intersection kernel SVM trained on a GPU.^[1] Dense sampling extracts local descriptors on a regular grid rather than only at sparse interest points. Vector quantization turns many local descriptors into a smaller set of visual words, making the representation manageable. Color SIFT features describe local gradient and color information. Spatial pyramid matching adds coarse layout information by pooling features over spatial cells at several scales. A histogram intersection kernel SVM compares feature histograms and classifies regions. Training on a GPU was part of the system and helped handle the computational load. The result was a complete localization stack: propose regions, describe them, pool the descriptions, and classify.

Fisher vectors and the pre-deep-learning curve

Fisher vector-based methods evolved over five years of the challenge and performed strongly in every ILSVRC from 2010 to 2014.^[1] Fisher vectors encode local descriptors by summarizing their distribution, and they often give a richer representation than simple visual-word histograms. The fact that these methods kept performing strongly across the challenge's first five years shows that the pre-deep-learning pipeline was not static: representation design, encoding, and classifier choices continued to improve. In 2011, the winning classification and localization systems fit this broader pattern of engineered features and efficient learning.

The 2012 turning point

The 2012 edition marked a turning point, when large-scale deep neural networks entered large-scale object recognition.^[1] The 2012 winner, SuperVision, trained a deep convolutional neural network with 60 million parameters using an efficient GPU implementation and a dropout trick.^[1] A convolutional neural network learns feature detectors directly from images rather than relying only on hand-crafted descriptors. Dropout randomly disables units during training, which helps prevent a large network from overfitting. The model had 60 million parameters, and the GPU implementation was central to training it. That result marked the entry of deep networks into large-scale object recognition; the 2011 results remain a clear snapshot of strong engineered pipelines just before that shift.