CIFAR-10

What it is

CIFAR-10 is a labelled image dataset used for training and testing machine-learning models that recognise what is in a picture. It holds 60000 colour images, each 32 by 32 pixels, divided into 10 classes of 6000 images each^[1]. The collection is split into 50000 training images and 10000 test images, and delivered as five training batches and one test batch of 10000 images apiece^[1]. The test batch is not a loose sample of the whole set: it contains exactly 1000 randomly selected images from each class, so every class carries the same weight in testing^[1].

The ten classes are airplane, automobile, bird, cat, deer, dog, frog, horse, ship and truck^[1]. The documentation describes them as completely mutually exclusive, so a single image belongs to exactly one of them^[1]. The line between the two vehicle classes is drawn on purpose: automobile covers sedans and SUVs, while truck is reserved for big trucks, so a picture of a lorry never counts as an automobile and a picture of a family car never counts as a truck^[1].

Where it came from

The dataset was created by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton^[1]. It is a labelled subset of a far larger collection of small pictures, the 80 million tiny images dataset, and a companion set, CIFAR-100, is drawn from the same source and made by the same three people^[1]. Krizhevsky's technical report on the work is dated 8 April 2009 and records 6000 examples for each of the 10 classes^[2].

What it looks like in use

The division into one test batch and five training batches matches how such a dataset is actually consumed. A model is shown the training images together with their labels and adjusts its internal parameters so that its answers move closer to those labels. How well it has done on those same images says little about new ones, so the held-out test batch is kept aside and used only to estimate how the model behaves on pictures it has never seen. Because the test batch holds exactly 1000 images per class, a score on it reflects performance across all ten categories rather than on whichever class happens to be easiest.

The pictures are small: 32 pixels on a side, so each image is a short list of numbers rather than a detailed photograph. What a model can pick up from them is coarse — the overall shape and colour of the subject rather than fine texture. That smallness is also practical: a dataset of this size can be held in memory and run through many experiments quickly, which is why a set like this is often used to try out an idea before scaling it up.

The documentation gives one concrete measurement of how hard the task is. A convolutional neural network — a model that learns small filters over local patches of pixels and stacks them into layers — reached 18 percent test error without data augmentation, and 11 percent with it^[1]. Data augmentation means generating altered copies of the training images, for example slightly shifted or flipped versions, so that the model sees more variation than the original 50000 pictures contain. The gap between the two numbers shows how much the extra variation is worth on this data.

Its place in the field

Because the split is fixed and the same for everyone, error rates reported by different methods on CIFAR-10 can be placed side by side; the 18 percent and 11 percent figures are reference points of exactly that kind, a level that a new method can be measured against^[1]. The dataset sits in a line of work on small, labelled image collections: a large pool of tiny pictures was filtered down into a compact set with ten clean, mutually exclusive categories, each represented by the same number of examples in training and in testing^[1][2]. That balance is what makes a single accuracy number meaningful, and it is what later work inherits when it uses the set.