Stability AI releases Stable Video Diffusion

Background

Stability AI's earlier models were image models built on Stable Diffusion. Stable Video Diffusion extended that work to video. Video is a sequence of frames, so a video model has to keep images consistent from one frame to the next, not simply produce one picture. The release was Stability's first generative video foundation model.

What happened

On 21 November 2023 Stability AI released Stable Video Diffusion as a research preview. ^[1] The preview contained two image-to-video models, generating 14 and 25 frames. They are known as SVD and SVD-XT. ^[1][2]

A research preview is a version released so that outside researchers can test and study a model before it is offered as a finished product, and it usually comes with limits on commercial use. Generating video is harder than generating a single image, because consecutive frames have to agree with one another in content, lighting and motion. A model that handles this well produces clips that look coherent rather than a series of unrelated pictures. The frame count sets how long a clip lasts at a given frame rate, which is why the 14-frame and 25-frame versions differ in length.

Stable Video Diffusion sits alongside the company's image models. For an image model, a single picture is the whole product; for a video model, the sequence of frames is the product, so the way motion unfolds over time matters as much as how each frame looks. Research previews are usually tested first by a small community, which helps developers find failure cases before a wider release.

Key details

Frame rates can be set between 3 and 30 frames per second. ^[1]

The code was published on GitHub and the weights on Hugging Face. ^[1]

Stability said the model was for research only and not yet for commercial use. ^[1]

Stability said external user-preference studies found that the model surpassed leading closed models. ^[1]

Impact

The release marked an early public step for Stability into generative video. Because it was a research preview, the company positioned it as a basis for further study rather than a finished commercial product. ^[1]