Stability AI launches Stable Audio

Background

Generating sound from a written description is one of the directions generative models moved into after text and images. Audio is difficult because it unfolds over time: a piece of music is continuous, its length is variable, and a good result has to match the description while still sounding like a complete piece of music or a coherent sound effect. Stable Audio is Stability AI's product in that direction, launched on 13 September 2023^[1][2].

What was released

Stable Audio was offered through a web interface at www.stableaudio.com^[3]. A user typed a descriptive text prompt and specified the length of audio wanted, and the model generated a track from it^[3].

At launch the product came in a free tier and a Pro subscription. The free version could generate and download tracks of up to 45 seconds^[1], while Pro delivered 90-second tracks that could be downloaded for commercial projects^[1]; a separate report described the free allowance as 20-second tracks, with Pro users getting up to 90 seconds^[2].

Stability AI described Stable Audio as the first tool to create 44.1 kHz music for commercial use using latent diffusion^[1]. 44.1 kHz is a sample rate — the number of times per second a sound is measured — and is a common specification for recorded and distributed music.

How it works

The model uses a latent diffusion architecture. Diffusion models learn by taking data to which noise has been added and learning to remove that noise step by step; once trained, they can start from pure noise and denoise their way to a sample. Latent diffusion runs this process inside a compressed representation rather than directly on the raw audio waveform, which makes the computation far cheaper, and decodes back to audio at the end.

In Stable Audio, text was not the only input. According to Stability AI, the architecture used audio as the generation target and conditioned it on text metadata as well as the audio file's duration and start time, which gave control over both content and length^[3]. That is why the interface asks for a description and a duration at the same time.

The training data came from a partnership with the AudioSparx music library and used licensed content^[2]. This matters for a music generation product: the rights status of the training material shapes whether generated output can be used commercially.

Significance

CEO Emad Mostaque said at launch that Stable Audio would empower music enthusiasts and creative professionals to generate new content with the help of AI^[3].

As a product, Stable Audio turned text-to-audio into a web tool that could be opened in a browser, distributed through a free allowance plus a subscription: shorter clips for free users, longer tracks and commercial-use downloads for paying ones^[1][2]. The use of licensed training content was part of how the product was presented^[2], which makes the rights position of the material relatively clear — important information for users who want to put generated tracks into commercial projects.