OpenAI announces DALL·E 2

Background

A text-to-image system takes a written description and returns a picture. Building one means connecting language to visual content: the model has to turn a sentence into something it can draw from. OpenAI's first system in this line, DALL·E, was a 12-billion-parameter model ^[1]. DALL·E 2 was the second ^[1].

The announcement

On 6 April 2022, OpenAI announced DALL·E 2 ^[1]. Two things about the announcement stood out beyond the images themselves: how the model was built, and how access to it would be handled.

How it worked

The distinctive design choice concerned what the model generates from. Rather than working from the caption directly, DALL·E 2 generated images from CLIP image embeddings using a diffusion decoder ^[2]. CLIP is a model trained to place an image and its caption close together in one shared vector space, so an embedding is a list of numbers that stands for the meaning of a picture or a piece of text. A diffusion decoder builds an image by starting from random noise and removing a little of it at each step, with the embedding steering every step.

That arrangement gave the system two capabilities beyond plain caption-to-image generation: image variations and language-guided edits ^[2]. Once an existing picture has been turned into an embedding, the model can generate new pictures from it, and an altered text description can push the result in another direction.

DALL·E 2 had 3.5 billion parameters, which made it smaller than the 12-billion-parameter original ^[1].

Results

OpenAI said DALL·E 2 produced images at 4x the resolution of DALL·E ^[4]. In its evaluations, people preferred DALL·E 2's output over DALL·E 1's 71.7% of the time when judging whether an image matched its caption, and 88.8% of the time when judging photorealism ^[4].

Access

Access widened in two steps. A paid beta opened on 20 July 2022 to 1 million people drawn from the waitlist ^[1][3]. From 28 September 2022 the waitlist was removed and anyone could sign up ^[1]. The result was that a system first shown as an announcement became, within a few months, something a very large number of people could try for themselves.

Safety

The announcement listed safety mitigations, among them limiting the ability to generate violent, hate or adult images ^[4]. These limits were described alongside the model rather than after the fact, and the staged rollout meant that use of the system grew in steps rather than all at once ^[1][3].

Why it matters

DALL·E 2 is best understood as a step in a line of systems rather than a single isolated release. It was the second text-to-image model from OpenAI, and it was markedly smaller than the first while producing images that evaluators preferred ^[1][4]. That combination is what makes the release worth remembering: the improvement came from a change in method — generating through a shared image-text embedding space and a diffusion decoder — as much as from adding parameters ^[2].

The release also set out the shape that later public deployments of generative image models tended to follow: a technical announcement with stated limits on what the model would draw, a waitlist-based beta for a limited group, and then open sign-up once capacity and safeguards allowed ^[1][3][4]. For readers looking at where today's image generators came from, DALL·E 2 is the point at which the approach moved from a research result to a service that a general public could use.