Cohere For AI releases Aya Expanse 8B and 32B

Background

Aya Expanse came from Cohere's research lab ^[1].

What happened

In October 2024 Cohere's research lab released Aya Expanse, multilingual models in 8B and 32B sizes ^[1]. The weights were released on Hugging Face under a non-commercial licence ^[2].

Key details

The models support 23 languages, including Japanese ^[1][3]. Cohere attributed their quality to data arbitrage, multilingual preference training and model merging ^[2]. Model merging combines the weights of separately trained models into one. The 32B model offers a 128k-token context window ^[3].

Cohere compared the models with Gemma 2, Llama 3.1, Mixtral 8x22B and Qwen 2.5 in judged win-rate tests, in which a judge model decided which answer was better ^[2].

Impact

The release gave researchers and developers two sizes of multilingual model covering 23 languages, with a long context window in the 32B version. Because the licence is non-commercial, the models were for research and other non-commercial use ^[2].