Visualizing and Understanding Convolutional Networks

The question behind the numbers

By 2013, large convolutional network models had shown impressive performance on ImageNet, the large-scale image classification benchmark ^[1]. What they had not produced was an explanation. Zeiler and Fergus note that there was no clear understanding of why these models performed so well, or of how they might be improved ^[1]. The gap has practical consequences: if the only feedback from an experiment is a final accuracy number, then design decisions — how many layers, how wide each one is, what filters to use — have to be guessed and then judged by that single number.

Looking inside: a visualization technique

A convolutional network is a stack of layers. An input image passes through them in order, and each layer rewrites what it receives into a set of feature maps that the next layer consumes; the end of the stack feeds a classifier that produces the prediction. Most of the computation happens in the intermediate layers, and those are the hardest to inspect, because their outputs are neither images nor answers.

The paper introduces a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier ^[1]. Rather than reporting only what the network decided, the technique turns the internal responses of the network into something a person can examine, so that one can ask what a given layer is responding to — a simple pattern, a texture, a larger structure — and how the classifier makes use of the resulting features ^[1].

Measuring instead of guessing: ablation

The second tool is an ablation study. Parts of a model are removed or altered, the change in performance is measured, and the difference is attributed to the piece that was changed. The authors perform such a study to discover the performance contribution of different model layers ^[1]. Instead of treating the network as one indivisible block that either works or does not, the ablation turns each layer into something whose value can be estimated.

Results

The two tools were not merely descriptive. The paper says they enabled the authors to find model architectures that outperform Krizhevsky et al. on the ImageNet classification benchmark ^[1]. In other words, the understanding gained from looking inside the network fed back into the design of a better network.

The paper also reports that the ImageNet model generalises well to other datasets. When the softmax classifier — the final layer that turns the network's features into scores for each class — is retrained on a new dataset, the model convincingly beats the then-state-of-the-art results on Caltech-101 and Caltech-256 ^[1]. Since only that last layer is relearned, the result says something about the features themselves: what the network learned from ImageNet is not tied to ImageNet's categories, and can be reused where the categories are different.

Why it matters

There are two takeaways. The first is diagnosis. Knowing what the layers do, and which of them actually carry performance, changes architecture design from guesswork into measurement; the paper's own improved architecture came out of exactly this loop of visualizing, ablating and redesigning ^[1]. The second is transfer: a model trained on one dataset carries a representation that remains useful on others ^[1]. Running through both is a framing the paper makes explicit — that a model which works well should also be one we can inspect and explain — and that framing is a large part of why the work is still cited.

Publication

The paper was first submitted to arXiv on 12 November 2013, with revisions recorded as v2 on 13 November 2013 and v3 on 28 November 2013 ^[1].