Stable Diffusion (v1)

What it is

Stable Diffusion is an open-weight text-to-image model: it takes a written prompt and produces an image. Stability AI announced its public release on 22 August 2022, after a first-stage release to researchers announced on 10 August^[1][4]. Because the weights were published openly, the model could be run on one's own hardware rather than only through a company's hosted service.

Where it came from

The method comes from latent diffusion, described in the paper "High-Resolution Image Synthesis with Latent Diffusion Models" by Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer, posted to arXiv in December 2021 and published at CVPR 2022^[3]. Hugging Face described the model as created by researchers and engineers from CompVis, Stability AI and LAION^[5]; Stability's launch announcement also credited Runway and the Machine Vision & Learning group at LMU Munich^[4].

How it works

A diffusion model learns to generate images by learning to reverse a noising process. During training, noise is added to images step by step and the network learns to remove it; at generation time the process starts from random noise and denoises, and an image appears. The latent-diffusion idea is to do this not on pixels but in the compressed latent space of an autoencoder: the autoencoder squeezes an image into a smaller representation, the diffusion process runs there, and a decoder turns the result back into a full image^[3]. Working in that smaller space is what makes training and sampling affordable.

The text prompt enters through cross-attention. In version 1 the prompt is encoded by a frozen CLIP ViT-L/14 text encoder with 123 million parameters, and the result is fed into an 860 million-parameter UNet through cross-attention layers^[2]. That is how words steer what the denoising process produces.

Training and released weights

The model was trained on a subset of the LAION-5B image-text database at 512x512 resolution^[2]. Several checkpoints were published, sd-v1-1 through sd-v1-4, with v1.4 the recommended public weights^[2]. Stability said training ran on its 4,000-A100 Ezra-1 cluster^[4].

What shipped with it

The release came under the CreativeML OpenRAIL-M license, which allows commercial and non-commercial use together with a set of use restrictions^[1]. The software package included an AI-based safety classifier, enabled by default, to filter unwanted outputs^[1]. Stability also offered a hosted app, DreamStudio, and recommended it for more control and rapid generation^[1].

Local use was meant to be within reach: Stability said the model runs on consumer GPUs with under 10 GB of VRAM^[4], and that final memory usage at release should be about 6.9 GB^[1].

Where it sits

Adoption was fast. By 17 October 2022, Stability said the model had been downloaded and licensed by more than 200,000 developers^[6]. It also became a starting point for later models: the successor lines include Stable Diffusion 2.0 in November 2022 and SDXL, whose paper in July 2023 describes a UNet backbone three times larger along with a second text encoder^[7].