Yi-Coder

What it is

Yi-Coder is a family of open-source code models released by 01.AI in September 2024. It comes in two sizes — 1.5B and 9B parameters — and each size is published both as a base model and as a chat model, giving four checkpoints in total.^[1][2] The models cover 52 major programming languages and accept context of up to 128K tokens, enough to place a fairly large source file, or a group of related files, in a single prompt.^[1]

How it was built

The 9B model was not trained from scratch. 01.AI took Yi-9B, a model it had already trained, and continued training it on an additional 2.4 trillion tokens of code.^[1][2] The training data had two parts: repository-level code taken from GitHub, and code-related data filtered out of CommonCrawl, a large public crawl of the web.^[1][2]

The phrase "repository-level" matters here. A great deal of code training data consists of isolated functions or short snippets, which teach a model syntax but not how the files of a real project refer to one another. Data organized by repository exposes the model to import statements, configuration files, tests, and the way a change in one file forces a change in another — closer to the setting in which a coding assistant is actually used. The 128K context window points the same way: it makes it possible to hand the model a body of existing code rather than a single question about one line.

Results

01.AI reported a set of benchmark numbers for Yi-Coder-9B-Chat. On LiveCodeBench it scored 23.4%, which the company described as the only result above 20% among models under 10B parameters; the comparison figures given are 22.3% for DeepSeek-Coder-33B-Instruct and 13.3% for CodeLlama-34B-Instruct.^[1] On the older and widely used HumanEval it scored 85.4%, and on MBPP 73.8%.^[1] 01.AI also said Yi-Coder was the first open-source code model to exceed 50% accuracy on CRUXEval-O.^[1]

Two further results test different skills. Aider's code-editing benchmark measures whether a model can apply a change to an existing codebase rather than write a function from a prompt; Yi-Coder-9B-Chat completed 54.1% of exercises, close to Qwen2-72B-Instruct at 55.6% and ahead of Codestral-2405 at 51.1%.^[1] Across seven program-aided math benchmarks — tasks where the model writes and runs programs to solve a mathematical problem — it averaged 70.3%, against 65.8% for DeepSeek-Coder-33B.^[1]

Where it sits

What makes these numbers notable is the size of the model producing them. Several of the systems Yi-Coder is compared against are three to eight times larger in parameter count, and in the Aider comparison the gap is far wider. The claim 01.AI is making is not that a 9B model beats the largest code models, but that careful continued training on the right kind of code data can put a small model in the same conversation as much bigger ones — which matters for anyone who has to run the model on their own hardware or serve it cheaply.

01.AI released the models as open source on 5 September 2024, China time.^[2] The Hugging Face blog post describing them carries a publication date of 4 September 2024.^[1] Both the 1.5B and 9B models, base and chat, were published, so the family spans from a size that fits on modest hardware to one aimed at serious coding assistance.