Stability AI releases Stable Audio 2.0

Background

Stability AI is a company that develops generative AI models. Stable Audio is the name of its family of models for sound and music. On 3 April 2024 the company released Stable Audio 2.0 ^[1], the version covered in this entry.

What happened

Stable Audio 2.0 was released on 3 April 2024 ^[1]. It generates tracks of up to three minutes at 44.1 kHz stereo ^[1]. The sample rate, 44.1 kHz, is the number of audio samples taken each second. It is the rate used for CD-quality audio, so the output sits at a familiar music quality level.

At launch the model was free to use on the Stable Audio website, and API access was to follow ^[1]. An API (application programming interface) lets other software send requests to a model and receive its output, so API access allows developers to build the model into their own products.

Key details

The release added audio-to-audio. Users can upload their own samples and transform them with text prompts, so an existing recording becomes the starting material for a new one ^[1].

On the technical side, Stable Audio 2.0 uses a diffusion transformer in place of the earlier U-Net ^[1]. A diffusion model creates output by starting from random noise and removing it step by step until the result matches the prompt. A U-Net is a neural network layout shaped like the letter U. A transformer is an architecture that lets every part of the input relate to every other part. The model also uses a highly compressed autoencoder ^[1]. An autoencoder converts data into a smaller internal representation and back again. Compressing the audio means the generation step works on much less data.

The training data was more than 800,000 licensed files from AudioSparx, whose artists could opt out of the training set ^[1]. Opting out means an artist can ask for their work to be excluded from training.

Impact

Stable Audio 2.0 is a dated record of one step in the Stable Audio line. It generated tracks of up to three minutes, edited user audio with text prompts, and was trained on licensed files ^[1]. Readers who compare it with later versions in this archive can use these points as the reference for the 2024 release.