Cambricon MLU 370

What the MLU 370 is

Cambricon released the MLU 370, its third-generation cloud AI chip, in 2021, together with accelerator cards built on it, and introduced its MLU-Link multi-chip interconnect at the same time ^[3]. Trade coverage of the launch dates the announcement to 3 November and names two cards, the MLU370-S4 and MLU370-X4 ^[4]. The chip is built on a 7 nm process and was Cambricon's first AI chip to use chiplet technology, packaging two AI compute dies into one chip ^[1].

Design and compute

A chiplet design splits the silicon into smaller dies that are fabricated separately and then packaged together, instead of building one large monolithic die. Cambricon says the MLU 370 integrates 39 billion transistors and delivers up to 256 TOPS at INT8 — integer arithmetic at 8 bits per number, the precision commonly used when quoting inference throughput. That figure is twice the compute of the second-generation MLU 270, according to the company ^[1].

The chip uses Cambricon's MLUarch03 architecture. This generation adds a module the company calls Supercharger, aimed at convolution efficiency — convolution being the operation at the heart of most image and video neural networks — and adds hardware fusion of multiple operators, so that several small operations can be executed as one rather than run separately ^[1].

Memory is one of the features Cambricon emphasises. It says the MLU 370 was the first cloud AI chip in China to publicly support LPDDR5 memory, the low-power memory standard more usually associated with phones, and that this gives three times the memory bandwidth of the previous generation, with 1.5 times the memory access efficiency of GDDR6 ^[1][2]. The claim is about bandwidth and access efficiency rather than raw peak arithmetic.

For larger jobs, the chip supports MLU-Link, a multi-chip interconnect intended for distributed training and inference, so several chips can be wired together and used as one bigger system ^[1]. Cambricon introduced MLU-Link alongside the chip itself ^[3].

Software: MagicMind

Cambricon paired the MLU 370 with MagicMind, which it describes as the industry's first inference engine built on MLIR graph compilation to reach commercial deployment ^[1]. MLIR is a compiler framework for describing programs as graphs of operations that can be transformed and optimised before they reach hardware; an inference engine built on it takes a trained model, rewrites it as such a graph, and maps the result onto the chip. The point of pairing chip and compiler is that the extra modules and fused operators in the architecture are only useful if the software can find and exploit them.

The video unit

The chip also carries a dedicated video unit. Cambricon says it can decode 132 streams of 1080p video or 10 streams of 8K video, and that it saves 42% of bandwidth when encoding at the same quality as the previous generation ^[1]. Those numbers matter for workloads that ingest many camera feeds, or that transcode video before and after inference — a common pattern in video analytics and in cloud media services.

Where it sits

Within Cambricon's cloud lineup, the MLU 370 is positioned as the cost-effective inference product, alongside the MLU 590 and MLU 690 ^[2]. Inference — running a trained model to produce answers — is generally cheaper and more latency-sensitive than training, so a part designed for it tends to trade peak arithmetic for memory bandwidth, video handling and interconnect. That is roughly the balance the MLU 370's features describe: LPDDR5 memory, a strong video decoder and multi-chip links, wrapped in a chiplet package that keeps two compute dies on one substrate.