Yi-34B and Yi-6B

What it is

Yi-34B and Yi-6B are a family of open large language models (LLMs) released by 01.AI, the Beijing company founded in 2023 by Kai-Fu Lee ^[1]. A large language model is a program trained on very large amounts of text to predict and generate language, which lets it answer questions, summarise documents and write code. The 34B and 6B in the names refer to the number of parameters, the adjustable numbers inside the network that are set during training. Yi-34B is the larger of the two, and Yi-6B is the smaller.

The Yi models are base models. They are trained to continue and understand text in general, and they can then be adapted into chat assistants. The family also includes chat versions, such as Yi-34B-Chat, which were built on the base models ^[1].

Origins

The base models Yi-6B and Yi-34B were released on 2 November 2023. Variants with a 200K context window followed on 5 November, and chat models, including Yi-34B-Chat and Yi-6B-Chat in plain, 4-bit and 8-bit quantised forms, arrived on 23 November 2023 ^[1]. A context window is the amount of text a model can read at once, and quantisation stores the model's weights with fewer bits to reduce memory use.

The company was at that time a young start-up. It had been founded in Beijing earlier in 2023, and the release came as 01.AI passed a valuation of US$1 billion about eight months after it started ^[4].

Main features

The models are bilingual, meaning they were pretrained from scratch on English and Chinese text, using 3 trillion tokens of training data. A token is a small unit of text, often a word or part of a word ^[1].

The base models were trained with a sequence length of 4K tokens, and they can be extended to 32K at inference, which is the stage when a finished model is used to generate answers. The 200K variants can handle about 400,000 Chinese characters in one input ^[1].

Yi uses the Llama architecture and can be loaded with the LlamaForCausalLM class used by Llama-style models. 01.AI states, however, that the models are not derivatives of Llama and do not use its weights ^[1]. Architecture refers to the design of the network, while weights are the learned numbers that give the model its behaviour, so the two are separate matters.

On benchmarks, Yi-34B ranked first among open-source pre-trained models in both English and Chinese on the Hugging Face Open LLM Leaderboard and on C-Eval, based on data available up to November 2023 ^[1]. Yi-34B-Chat ranked second on AlpacaEval, behind GPT-4 Turbo, with data up to January 2024, and it ranked first among open models on MMLU, CMMLU, BBH and GSM8k ^[1]. AlpacaEval, MMLU, CMMLU, BBH and GSM8k are benchmarks that test instruction following, general knowledge, Chinese knowledge, reasoning and grade-school mathematics respectively.

On 7 March 2024 the Yi-34B-200K long-context version was improved. Its score on the Needle-in-a-Haystack test rose from 89.3% to 99.8%. That test hides a short fact inside a long text and asks the model to find it. The improvement used a long-context data mixture of 5 billion tokens ^[1].

Yi-9B, released on 6 March 2024, was trained further from Yi-6B on 0.8 trillion more tokens. 01.AI said it was the best overall among open models of similar size ^[1].

Licence

At launch the models were open for research and non-commercial use under 01.AI's own licence ^[3]. The code and weights are now distributed under the Apache 2.0 licence ^[1]. Apache 2.0 is a permissive open-source licence that allows use, modification and redistribution, including commercial use, as long as the licence notices are kept.

Its place in development

The Yi-34B release was a notable example of a Chinese open model appearing on the same public leaderboards as models from the United States. Its technical report, Yi: Open Foundation Models by 01.AI, was published on arXiv in March 2024 ^[2][1]. 01.AI said that in the first month Yi-34B took nearly half of the trending-model slots in the open-source community, and that the model was later added to NVIDIA's playground, a hosted environment for trying models ^[5]. These figures come from the company and show how the model was presented to developers. Yi-34B also served as a base for other models in the Yi line, including the Yi-VL vision-language model.