Runway Gen-2

What Gen-2 is

Runway Gen-2 is a multimodal AI system that generates new video from text, images or video clips ^[1]. It is not a single tool but a set of generation modes collected in one product, offered in the Runway web app ^[1].

Where it came from

Runway, the company behind it, has been described as a Google-backed startup that helped develop the Stable Diffusion image generator ^[5]. Gen-2 was introduced in March 2023 ^[2]. For its first months it was reachable only through limited, waitlisted access ^[5]; it became publicly available in June 2023 ^[2].

The step Gen-2 marked was the addition of text-to-video generation to the video-to-video abilities of the earlier Gen-1 model, which put it among the first commercially available text-to-video models ^[2]. The difference between the two tasks is substantial: a video-to-video system takes an existing clip and re-renders it, while a text-to-video system has to invent motion, timing and scene structure from a written prompt alone.

The modes

Runway's own description lists three new modes — Text to Video, Text + Image to Video and Image to Video — alongside the modes carried over from Gen-1: Stylization, Storyboard, Mask, Render and Customization ^[1]. Counting the new and the inherited modes together, the Gen-2 page lists eight in total ^[4]. The system has likewise been described as a multimodal system supporting text to video, text plus image to video, and image to video variations ^[4].

Because the older modes were kept alongside the new ones, the same product covers both starting a clip from a prompt or a still image and transforming footage that already exists ^[1][4].

Reception

When Gen-2 opened to the public in June 2023, TechCrunch's review argued that it showed the limitations of text-to-video technology at the time ^[3]. To let people judge for themselves, Runway gave around 100 seconds of free video generation for trying the model ^[5].

Place in the line

Gen-2 was followed by Gen-3 Alpha in June 2024 ^[2]. Within Runway's sequence of models, Gen-2 is the point at which the product moved from transforming existing video to generating video from text and images as well ^[2].