Aya 101

What it is

Aya 101 is a massively multilingual model from Cohere For AI, the nonprofit research lab of Cohere. It follows instructions in 101 languages, and more than half of those languages are considered lower-resourced, meaning that there is relatively little written text available for training ^[1]. It has 13 billion parameters, the adjustable numbers that store what a model has learned, and its weights were released on Hugging Face under the Apache 2.0 licence ^[2][3].

Origins

The model is described in a paper first posted to arXiv on 12 February 2024 ^[1]. Its training data was limited to the 101 languages that the mT5 model already supports ^[2].

Aya 101 was built as an instruction-tuned model, meaning it was further trained on examples of instructions and good answers so that it responds to requests. Its builders published the instruction datasets alongside the model ^[1].

Main features

The authors' central aim was breadth, with 101 languages covered. The authors report that it outperforms mT0 and BLOOMZ on most tasks while covering twice as many languages ^[1].

To test the model, the team built new evaluation suites spanning 99 languages. These included human evaluation and simulated win rates, which compare the model's answers against those of other systems ^[1].

Place in development

In May 2024 Aya 101 was succeeded by Aya 23, which traded breadth for depth by covering 23 languages ^[1].