Zhipu open-sources the CogVideoX video model

Background

Zhipu's open-source video work began earlier with CogVideo, which was open-sourced on 19 May 2022 and described as the first open-source large Transformer-based text-to-video model.^[1] A text-to-video model turns a written description into a short video clip, and the Transformer is the neural network design that most large language and vision models now share.

In July 2024 Zhipu launched Ying (清影), a text-to-video product. Ying generated six-second clips in about 30 seconds.^[2] Ying was a commercial product, and the open-source models were released later that summer.

What happened

On 6 August 2024 Zhipu AI and Tsinghua University released CogVideoX-2B, the first public model in the CogVideoX family. CogVideoX-5B followed on 27 August 2024.^[1] The release made the open-source version of the video generation model behind Ying available to anyone who wanted to download it.^[1]

Key details

Both CogVideoX-2B and CogVideoX-5B generate video at 720×480 pixels.^[1] The two versions carry different licences. CogVideoX-2B is released under Apache 2.0, a permissive open-source licence that allows broad reuse. The 5B models use a separate CogVideoX licence.^[1] Because the terms differ, a user who wants to run one of these models should read the licence attached to that version before using it.

The family is described as descending from CogVideo, the 2022 project that opened the line.^[1] Its name records that lineage: CogVideo became the starting point, and CogVideoX is the version that carries Zhipu's text-to-video work forward.

Impact

The release gave researchers and developers a public, downloadable text-to-video model from a Chinese lab, next to the commercial product Ying. Open-sourcing lets people inspect the model, run it on their own hardware and build on it, within the limits set by each version's licence. The two-tier release, with a permissive licence for the 2B model and a separate licence for the 5B models, shows that the terms of a model family can differ even when the models come from the same project.