AI21 Labs releases Jamba 1.5 Mini and Large

Background

The original Jamba, released in March 2024, combined Mamba state-space layers with Transformer attention in an open-weights model. By August 2024 the company was ready to publish a follow-up in the same design family, now aimed at instruction-following use. A hybrid Transformer-Mamba model mixes the two kinds of layers, with attention for relating distant parts of a text and Mamba layers handling most of the sequence at lower cost.

What happened

On 22 August 2024 AI21 Labs published Jamba 1.5, a pair of hybrid Transformer-Mamba models ^[1]. The models are Jamba 1.5 Mini and Jamba 1.5 Large. The paper describing them, "Jamba-1.5: Hybrid Transformer-Mamba Models at Scale", was posted to arXiv on the same date ^[1].

Key details

Jamba 1.5 Large has 94 billion active parameters, and Jamba 1.5 Mini has 12 billion ^[1]. Active parameters are the ones used for each token; in a mixture-of-experts model this number is smaller than the total.

Both models support an effective 256k-token context ^[1]. A context window of this size lets a model work with very long documents in a single prompt.

AI21 open-sourced ExpertsInt8, a quantisation technique. With it, the Large model can be served on eight 80 GB GPUs ^[1]. Quantisation reduces the memory that a model needs by storing its numbers at lower precision, which is what makes running a model of this size on one machine practical.

The weights are available under the Jamba Open Model License ^[2]. The licence governs how the models may be used, and readers should check its terms for any particular use.

In September 2024 AWS began offering Jamba 1.5 on Amazon Bedrock ^[3].

Impact

The release made a larger member of the Jamba family available with open weights, along with a method for running it on less hardware. It also extended the family's availability through AWS. The next version, Jamba 1.6, is covered in its own entry.