Very Deep Convolutional Networks for Large-Scale Image Recognition

What the paper set out to do

Convolutional networks (ConvNets) are a standard architecture for recognising what a photograph contains: stacks of filters slide over the image, each layer turning the previous layer's feature maps into slightly more abstract patterns, and the last layers turn those patterns into a class label. A natural question follows — does adding layers keep helping, and how far can depth be pushed before the design stops paying off? The paper addresses that question by evaluating networks of increasing depth in a controlled setting.^[1]

How it was done

The design choice at the centre of the work is filter size: the architecture uses very small 3×3 convolution filters throughout.^[1] With that fixed, the authors trained configurations of increasing depth and compared them, eventually pushing depth to 16–19 weight layers, which the abstract reports as a significant improvement over prior-art configurations.^[1] The paper was first submitted to arXiv on 4 September 2014, and a final revision followed on 10 April 2015.^[1]

Keeping the filters small is what makes the comparison meaningful. A 3×3 filter sees only a tiny neighbourhood of the image at a time, so a single layer captures very local patterns; depth is what lets the network combine those local patterns into larger structures. Holding filter size fixed means that when the results change, depth is the variable that changed.

Results

Those findings were the basis of the team's ImageNet Challenge 2014 submission. In that challenge the team took first place in the localisation track and second place in the classification track.^[1] The full text spells out the classification numbers: 7.3% test error with an ensemble of 7 models, later reduced to 6.8% with an ensemble of 2 models.^[2] The authors call that competitive with the classification winner, GoogLeNet, at 6.7% error, and describe it as a substantial improvement over the 2013 winning Clarifai submission, which reached 11.2% with outside training data and 11.7% without it.^[2]

Reading the numbers

An ensemble means training several networks and combining their predictions. It usually costs more computation at test time than running a single model, but tends to lower the error rate. Two details are worth keeping in mind. The headline classification figures come from ensembles rather than from one network, and the best of them, 6.8%, used only 2 models — fewer than the 7 models behind 7.3%.^[2] The comparison with the 2013 winner also separates two things that are easy to confuse: the error obtained when training data from outside the challenge set is allowed, and the error when it is not.^[2]

Why it mattered

The recipe is unusually simple to state — many layers, all filters 3×3, depth varied systematically — and it showed that depth itself accounts for a large part of the gain over earlier configurations.^[1] Just as important for later work, the authors made their two best-performing models publicly available.^[1] Released weights let other researchers check the reported numbers, start from a trained network instead of from scratch, and compare a new architecture against a fixed, well-documented baseline. That is a quieter contribution than a leaderboard position, but it is the one that keeps a model useful after the challenge is over.