Aya Expanse

What it is

Aya Expanse is a family of multilingual, open-weight language models released by Cohere For AI in October 2024, in two sizes, 8B and 32B^[3][1]. The model card for the larger version lists 32 billion parameters^[1]. Open weights means the trained weights can be downloaded and run by others, rather than only being reachable through a hosted interface.

The family covers 23 languages: Arabic, Chinese, Czech, Dutch, English, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Korean, Persian, Polish, Portuguese, Romanian, Russian, Spanish, Turkish, Ukrainian and Vietnamese^[2]. The list mixes languages with very large speaker populations and large digital footprints, such as English, Chinese and Spanish, with languages that have long been thinly represented in training data and benchmarks, such as Greek, Hebrew, Romanian and Ukrainian. Covering them with a single set of weights, rather than one model per language, is the organising idea of the release.

How it was built

Cohere For AI describes the models as combining Cohere's Command family of pre-trained models with four research techniques: data arbitrage, multilingual preference training, safety tuning and model merging^[1]. These correspond roughly to four different stages of building a model.

Data work decides what the model reads while it is being trained. Preference training tunes a model towards answers that a judge rates as better, instead of only training it to predict the next token. Safety tuning applies the same kind of adjustment to harmful or policy-violating outputs. Model merging takes separately trained models and combines their weights into one, which is a cheaper route than training a single large model from scratch. How much each of these contributed is a question for the technical report rather than for the model card.

Context window and output length

The 32B model has a context window of 128k tokens and can produce up to 4k tokens of output^[2]. A token is roughly a word or a piece of a word. A 128k-token context means a user can put a long document — a book chapter, a large source file, a full transcript — into a single request. The output limit is a separate constraint: it bounds how long one answer can be.

Evaluation

The team evaluated Aya Expanse with win-rate comparisons. In this setup a judge model is shown two answers to the same prompt and asked which is better, and a model's score is how often its answer is preferred. Here the judge was GPT-4o, and the comparison set was Gemma 2 27B, Llama 3.1 70B, Mixtral 8x22B and Qwen 2.5 32B^[1]. The evaluation used the Arena-Hard-Auto dataset translated into the 23 supported languages^[4]. According to the arXiv paper, Aya Expanse 32B outperforms Llama 3.1 70B, a model with twice as many parameters^[4]. Win rates produced by a model judge are a proxy measure; they record how often one answer is preferred by that judge.

Licence and availability

The models are released under CC-BY-NC 4.0, a non-commercial licence^[1][2]. The model card states that use is governed by CC-BY-NC and also requires adhering to Cohere Labs' Acceptable Use Policy^[1]. The weights can therefore be downloaded and used for research and other non-commercial work. The model is also available through Cohere's Chat API under the name c4ai-aya-expanse-32b^[1][2].

Technical report

The technical report, "Aya Expanse", was posted to arXiv in December 2024^[4].

Where it sits

Aya Expanse 32B serves as the language model of Aya Vision 32B, released in March 2025^[1]. In 2026 Cohere added Tiny Aya, a 3.35B-parameter family for 70 languages designed to run on device^[1][5]. The later releases take the line in two directions: Aya Vision pairs the language model with another kind of input, while Tiny Aya is far smaller, covers more languages and is meant to run locally rather than as a large hosted model.