GoogLeNet and VGG dominate ILSVRC 2014

Background

The ImageNet Large Scale Visual Recognition Challenge, or ILSVRC, is an annual competition in which research groups train image-recognition systems and have their predictions scored on a shared benchmark. Two of its tracks matter here. In the classification track a system has to say what object an image contains; in the localisation track it has to draw a box around that object. Classification entries are ranked by top-5 error, the share of images on which the correct label is missing from the system's five best guesses, so a lower number is better. By 2014 more than 50 institutions were taking part in the challenge.^[3]

What happened

The 2014 edition was won in classification by GoogLeNet. The GoogLeNet paper reports that its submission reached 6.67% top-5 error on both the validation and the test data, and ranked first.^[1]

The gap at the top was small. The same paper's results table lists GoogLeNet first at 6.67%, VGG second at 7.32% and MSRA third at 7.35%, so the leading three entries were separated by less than one percentage point.^[1]

The localisation track went differently. The VGG team reports that it took second place in classification and first place in the localisation track at ILSVRC 2014.^[2] Wikipedia's account of the challenge agrees on the split: the winning classification entry in 2014 was GoogLeNet and the winning localisation entry was VGGNet.^[3]

Key details

GoogLeNet's contribution was a whole architecture rather than a single add-on. The paper describes it as a 22-layer-deep Inception network that increases both the depth and the width of the network while keeping the computational budget constant.^[1] Depth and width pull against budget: stacking more layers (depth) or adding more filters per layer (width) normally costs more arithmetic, so the design question becomes how to spend a fixed amount of computation more effectively. The paper presents the network as a way of getting more depth and width out of that fixed budget.^[1]

The VGG network took a different route through the same competition: it did not win classification but did win localisation.^[2][3] The two tracks ask different things of a system — naming an object versus locating it — and the 2014 results show a system can be strong at one and not the other.

Why it matters

A challenge like ILSVRC works as a scoreboard: it lets very different designs be measured on the same data with the same metric. In 2014 that comparison put two approaches at the front — GoogLeNet's budget-conscious Inception design and the VGG network — with MSRA close behind.^[1] The near-tie at the top is a reminder that progress in the field was not a single team's story.

The split between the two winners is the other lasting detail. Being able to say what is in an image and being able to say where it is are related but distinct abilities, and the 2014 challenge scored them in separate tracks with separate winners.^[2][3] For anyone reading the results later, the table also makes the scale of the field visible: with more than 50 institutions entering, the leading scores sit within a narrow band at the top rather than standing alone.^[1][3]