Learning Multiple Layers of Features from Tiny Images (CIFAR-10/100)

The problem it addresses

Training a program to recognise what is in a picture requires examples whose answers are already known: the model has to be told, for many images, which category each one belongs to before it can learn a mapping from pixels to category names. CIFAR-10 and CIFAR-100 were assembled to supply that material. Both are labeled subsets of the 80 million tiny images dataset, created by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton ^[1]. The construction of the datasets is documented in Krizhevsky's 2009 technical report, *Learning Multiple Layers of Features from Tiny Images* ^[1].

How the dataset is built

CIFAR-10 consists of 60,000 colour images, each 32×32 pixels, spread over 10 classes with 6,000 images per class ^[1]. Of these, 50,000 are training images and 10,000 are test images, divided into five training batches and one test batch, each holding 10,000 images ^[1]. The test batch contains exactly 1,000 randomly selected images from each class, and the five training batches together contain exactly 5,000 images from each class ^[1].

The 10 classes are airplane, automobile, bird, cat, deer, dog, frog, horse, ship and truck, and they are completely mutually exclusive ^[1]. Mutual exclusivity matters because a label has to be a single answer: an image is either an automobile or a truck, never counted as both, so the correct response for every image is unambiguous. The even distribution matters for a different reason: with exactly 6,000 images per class, a classifier cannot score well simply by favouring the classes that appear more often.

The 32×32 size is the other design decision that shows in the numbers. A single image is small enough that storing 60,000 of them and running many training experiments over them stays cheap, which is what makes the collection usable as a repeated reference point rather than a one-off test.

Baseline results

The report records what a convolutional neural network (a network that slides small learnable filters over the image rather than treating every pixel as an independent input) achieved on CIFAR-10: 18% test error without data augmentation and 11% with it ^[1]. Data augmentation means enlarging the training set by making altered copies of the images already there, so the network sees more varied versions of the same examples. The gap between the two figures is itself informative: it shows how much of the difficulty lay in the limited number of distinct training images rather than in the classes themselves.

Why it matters

Taken together, the properties of CIFAR-10 make it a usable yardstick. The training and test images are fixed in advance, so a 10,000-image test batch means the same thing for everyone who uses it. Each class contributes exactly 1,000 test images, so overall accuracy is not dominated by whichever class happens to be easiest. And the classes are defined so that no image belongs to two of them. A number such as "11% test error" therefore has a clear meaning that can be compared with another method's number on the same split.

Where it sits

The technical report does two things at once: it specifies how the data is organised, and it reports baseline error rates obtained with a convolutional network ^[1]. A reader coming to the dataset can therefore see not only the images and their labels but also a reference level of performance, and can judge new results against the same fixed training batches, the same test batch and the same per-class counts ^[1]. CIFAR-100 belongs to the same family, a further labeled subset drawn from the 80 million tiny images collection by the same three people ^[1].