Progressive Growing of GANs for Improved Quality, Stability, and Variation

The problem it addresses

Progressive Growing of GANs for Improved Quality, Stability, and Variation is a paper by Tero Karras, Timo Aila, Samuli Laine and Jaakko Lehtinen. It was first submitted to arXiv on 27 October 2017 under the identifier 1710.10196, and it describes a new training methodology for generative adversarial networks (GANs) ^[1].

A GAN is built from two networks trained against each other. The generator takes random input and turns it into an image; the discriminator looks at an image and judges whether it came from the training data or from the generator. Each improves by pushing against the other: the generator tries to produce images the discriminator cannot reject, and the discriminator tries to catch the fakes. This works reasonably well at small sizes, but the two-player arrangement is delicate. If one side becomes much stronger than the other, the signal the other receives becomes useless, and training can stall or diverge. The problem gets worse as images get larger, because the discriminator's task becomes harder and the generator has to control many more pixels at once.

Growing instead of starting big

The paper's key idea is to grow both the generator and the discriminator progressively. Training starts at low resolution, with shallow networks that only have to get the rough layout of an image right. As training proceeds, new layers are added to both networks, so that they model increasingly fine detail ^[1]. Because the two networks grow together, the discriminator does not run far ahead of the generator; the difficulty of the game rises step by step, and each new layer is trained on top of features that have already settled. In effect, the paper treats the size of the network not as a fixed design choice but as a schedule that runs alongside training.

The authors report that this speeds training up and greatly stabilises it, allowing images of unprecedented quality, such as CelebA images at 1024x1024 ^[1]. CelebA is a widely used dataset of face photographs; at that resolution each image contains more than a million pixels, so the generator has to produce convincing fine structure such as hair, skin and background detail rather than only a face-shaped blob.

The authors also propose a simple way to increase the variation in generated images ^[1]. Variation matters because a generator can otherwise settle into producing a narrow set of outputs that please the discriminator, a failure sometimes called mode collapse: the samples look good individually but do not cover the range of the data.

Results

The method reaches a record inception score of 8.80 in unsupervised CIFAR-10 ^[1]. The inception score is a benchmark that rewards generated images which look like clear, recognisable objects and which are diverse; CIFAR-10 is a standard image benchmark, and the unsupervised setting means the method is not given class labels. A record on this benchmark is a quantitative complement to the visual results at high resolution.

Why it matters

Two of the practical obstacles to GANs have been instability and the difficulty of scaling to high resolution. The paper addresses both with the same mechanism, and adds a separate, cheap fix for variation. The result is a recipe in which resolution is raised gradually and quality, stability and variation are improved together rather than traded off one against another.

Limits and place in the field

The contribution is a training methodology rather than a new dataset or benchmark: it changes how, and in what order, the networks are trained ^[1]. Its headline numbers are benchmark scores, which summarise quality and diversity only imperfectly, and the high-resolution demonstration uses face photographs, a relatively constrained kind of image. Training at high resolution also remains expensive in computation. Still, the paper makes a point that carries beyond any single score: what a generative model learns depends heavily on the order in which it is asked to learn it.