Stable Video Diffusion

What it is

Stable Video Diffusion (SVD) is an image-to-video generative model: given one still image, it produces a short moving clip. Stability AI released it on 21 November 2023 and described it as the company's first foundation model for generative video, built on Stable Diffusion^[1]. It arrived as a research preview rather than a finished product, and it shipped as two models rather than one^[1].

The two models and their output

The smaller of the pair generates 14 frames and is named SVD; the larger generates 25 frames and is named SVD-XT^[2]. Frame rate is not fixed — it can be set anywhere between 3 and 30 frames per second^[1]. At its usual settings the result is a short clip of roughly 2 to 4 seconds at 576x1024 resolution^[2]. Because frame count and frame rate are both adjustable, the same model can produce a smoother, shorter clip or a longer, choppier one, but the output stays within the range of a few seconds.

Data and training recipe

The paper published alongside the model treats data as the central problem. It argues that a well-curated pretraining dataset is necessary for generating high-quality video^[3]. To get there, the authors present a systematic curation process, including captioning and filtering strategies, and use it to train a strong base model^[3]. The paper also identifies and evaluates three different training stages for video latent diffusion models^[3]. Beyond the image-to-video models that were released, the paper describes training a text-to-video model that it says is competitive with closed-source video generation^[3].

How it was released

The code sits in the Stability-AI/generative-models repository on GitHub, and the weights are published on Hugging Face under stabilityai/stable-video-diffusion-img2vid-xt^[1]. The model can be fine-tuned for downstream tasks, with multi-view synthesis from a single image given as an example^[1]. Stability said external user-preference studies found that the model surpassed leading closed models at the time of release^[1]. At the same time, the company stated that the model was not intended for real-world or commercial applications at that stage^[1].

Where it sits

Stable Video Diffusion extends the open-weights approach Stability AI had already taken with Stable Diffusion from images to short video. Its position at release is defined by two things stated plainly by the company: it was a research preview, and it was benchmarked through external user-preference studies rather than presented as a deployable product^[1]. The output format — a few seconds at modest frame counts — and the explicit note that commercial use was out of scope at that stage mark the model as a step in extending image generation to motion rather than a finished tool^[1][2]. The fine-tuning support for tasks such as multi-view synthesis and the open publication of code and weights point to the intended use: research and further training rather than end-user applications^[1].