Alibaba releases the Qwen1.5 open model family

Background

Qwen1.5 was the next release in Alibaba's Qwen line of large language models.^[3]

What happened

The Qwen team introduced Qwen1.5 in a blog post dated 4 February 2024. The Qwen repository lists the release as 5 February 2024.^[1]^[2] Alibaba released the family just before Chinese New Year. The release included base and chat models in a range of sizes, from 0.5 billion to 72 billion parameters. The range was later extended to 110 billion parameters, and the release also included a mixture-of-experts model.^[1]

Key details

Every model in the family supported a context length of up to 32,768 tokens. A token is the unit of text a model reads and writes, roughly a word or part of one.^[1] The team published benchmark results for its largest base model, Qwen1.5-72B. It scored 77.5 on MMLU, 84.1 on C-Eval, 79.5 on GSM8K and 41.5 on HumanEval, and it outperformed Llama2-70B on all of the benchmarks listed.^[1]

The chat model, Qwen1.5-72B-Chat, scored 8.61 on MT-Bench. Its 27.18 win rate on AlpacaEval 2.0 was ahead of Claude-2.1 and GPT-3.5-Turbo-0613, but still well behind GPT-4-Turbo.^[1]

The models were built for developers who wanted to run them in many tools. They were integrated directly into Hugging Face transformers 4.37.0 without trust_remote_code, and they had support in vLLM, SGLang, Ollama, LM Studio and llama.cpp. Quantized versions were offered in Int4 and Int8 formats, using the GPTQ, AWQ and GGUF methods.^[1] Quantization stores a model's numbers at lower precision, which reduces the memory needed to run it.

A base model is trained to predict the next piece of text, while a chat model is tuned further so that it answers questions and follows instructions in a conversational way. Benchmark scores such as MMLU and MT-Bench are one way to compare models, but each test measures only the tasks it contains, so they are partial evidence rather than a complete ranking.