01.AI releases the open Yi-34B and Yi-6B models

Background

In 2023 a number of Chinese companies began building large language models, the systems that power chatbots and many other AI tools. 01.AI was founded in Beijing in March 2023 by Kai-Fu Lee, who had earlier founded Microsoft Research China and served as president of Google China ^[1]. The company had passed a valuation of US$1 billion about eight months after it started, following a funding round that included Alibaba's cloud unit ^[3].

Open models matter to researchers and developers because they can download the weights, run the model on their own hardware and study how it behaves. The most widely used place to publish such models is Hugging Face, a platform where developers share model weights, datasets and model cards. A model card is a document that describes a model's training, intended use and evaluation results.

What happened

On 2 November 2023 01.AI released the Yi-6B and Yi-34B base models. Five days later, on 5 November, it followed up with versions that can handle a 200K context window, which means the model can read a much longer input in one go ^[1]. The models were pretrained on 3 trillion tokens of English and Chinese data, and the company presented them as bilingual ^[1].

The Yi-34B model ranked first among pre-trained open models on Hugging Face's Open LLM Leaderboard and on C-Eval. The ranking placed it ahead of Meta's Llama 2 ^[2].

The models were initially open for research and non-commercial use, which meant companies could study them but could not use them to build paid products ^[2]. The licence of the code and weights was later changed to Apache 2.0, which permits commercial use ^[1].

On 23 November 2023 chat models followed, and 01.AI updated its licence agreement to version 2.1 on the same day ^[1]. Base models can continue text but are not tuned to hold a conversation, so chat versions are what most people encounter when they try a model as an assistant.

Key details

Two facts about the company's funding were reported alongside the release. Kai-Fu Lee said 01.AI had passed a valuation of US$1 billion after a funding round that included Alibaba's cloud unit, about eight months after he started the company ^[3]. A valuation of more than US$1 billion is what makes a start-up a unicorn.

The release therefore marked two milestones together: a set of open models that ranked first on public tests among open pre-trained models, and a company that had grown into one of the best-funded AI start-ups in China in a short time.

Impact

For readers who are new to AI, the Yi-34B release shows how open models compete. Developers judge a model by its results on shared tests such as C-Eval, which measures Chinese knowledge, and by how easily they can use the weights. The Yi family grew from this release. The Yi-6B and Yi-34B models were followed by long-context, chat, vision, coding and larger models, and the Yi-34B base became the foundation for several later Yi releases.

The event also shows how licences shape adoption. A model that is open for research but restricted for commercial use reaches a different set of users than one released under Apache 2.0. When the licence changed, companies could build products on the models, and this step widened the range of people who could use them.

Finally, the release shows the role of public leaderboards. Results on the Open LLM Leaderboard and C-Eval were the primary way that people outside the company compared its models with others when the models were new. The records of those rankings reflect the state of open models in late 2023, and readers should check later results on the same boards to see how the field has changed.