Meta MTIA v2

What it is

Meta MTIA v2 is the second generation of Meta's in-house AI chip, the Meta Training and Inference Accelerator. Meta published it on 10 April 2024 ^[2]. It is not a chip sold to outside customers: Meta designs it to serve the ranking and recommendation models running on its own systems ^[1].

Where it came from

The name marks its position as a successor. After a first-generation MTIA, Meta presents the second generation as an upgrade along the same line rather than a fresh start: it says the new chip more than doubles the compute and memory bandwidth of its previous solution ^[1]. On compute specifically, Meta puts the gain at 3.5 times the dense compute and 7 times the sparse compute of MTIA v1 ^[1]. The power budget moved with it — a 90 W thermal design power, up from 25 W in the first generation ^[1].

How it is built

According to Meta's specification table, the chip is manufactured on TSMC's 5-nanometer process and runs at 1.35 GHz ^[1]. Its compute is organised as an 8 by 8 grid of processing elements ^[1] — 64 units working in parallel, a common way to spread matrix arithmetic across a chip. Feeding them is 256 MB of on-chip memory running at 2.7 TB/s, backed by 128 GB of off-chip LPDDR5 at 204.8 GB/s ^[1]. Meta lists 708 teraflops for INT8 GEMM with sparsity and 354 teraflops dense ^[1].

Coverage of the launch quoted slightly different figures from the same neighbourhood of the table: Securities Times reported the move to TSMC's 5 nm process, 256 MB of on-chip memory and a 1.3 GHz clock, and wrote that in Meta's early tests the new chip was three times as fast as MTIA v1 ^[2].

How it is deployed

Meta says the chip is hosted in a rack-based system that holds up to 72 accelerators, spread across three chassis of 12 boards with two accelerators each ^[1]. The accelerators connect to their hosts over PCIe Gen5 ^[1]. What Meta describes is therefore a system rather than a bare die: boards, chassis, host links and a rack-level count of accelerators, which is the form in which such a chip is actually put to work.

Why it matters

The chip is less interesting as a benchmark entry than as a marker of how large internet companies build silicon for their own workloads. Meta states plainly that the new MTIA is designed to serve its ranking and recommendation models ^[1] — models that have to be evaluated against very large numbers of requests and are therefore sensitive to memory bandwidth and to how many accelerators can be packed into a rack ^[1]. Figures such as 354 teraflops dense and 2.7 TB/s of on-chip memory only mean something against that kind of workload; they are not the same measure as headline numbers for chips built to train very large models. The generation-to-generation step is visible in the specifications themselves: a jump from 25 W to 90 W of thermal design power ^[1], a 3.5-fold dense and 7-fold sparse compute increase over MTIA v1 ^[1], and a memory hierarchy that pairs a fast 256 MB on-chip pool with a much larger off-chip LPDDR5 pool ^[1]. Read together, they describe a chip sized for a specific class of models rather than for general-purpose computing.