AMD Instinct MI250X

What it is

The AMD Instinct MI250X is a data-center accelerator: a GPU designed for computation rather than for drawing images, and sold as a component to be built into servers. AMD lists 8 November 2021 as its launch date^[1]. The chip is built on the CDNA 2 architecture and manufactured by TSMC on a 6 nm FinFET process^[1]. CDNA is AMD's family of GPU designs aimed at data-center workloads, and the architecture's white paper sets out the memory and cache hierarchy that feeds its compute units^[2].

Computing resources

The MI250X contains 220 compute units and 14,080 stream processors, with a peak engine clock of 1,700 MHz^[1]. A compute unit is the repeated building block of a GPU — a bundle of arithmetic lanes, registers and local storage — and the stream processors are the individual lanes inside it that carry out the arithmetic. Performance is usually quoted as a peak rate: how many operations the chip could complete per second if every lane were busy and no time were lost waiting for data.

AMD's rated peaks are 383 TFLOPS for FP16 and BF16^[1]; 47.9 TFLOPS for FP32 and 95.7 TFLOPS for FP32 matrix operations^[1]; 47.9 TFLOPS for FP64 and 95.7 TFLOPS for FP64 matrix operations^[1]; and 383 TOPs for INT4 and INT8^[1].

Two patterns in those numbers are worth noticing. The matrix figures are exactly double the ordinary (vector) figures, which reflects dedicated matrix hardware: instead of handling one pair of numbers per instruction, it consumes a small tile of them at once. And FP64 — double precision, the format used when results must stay accurate across long chains of calculation, as in physical simulation — is rated at the same 47.9 TFLOPS as FP32, not at a fraction of it. On graphics-oriented GPUs, double precision typically runs far slower than single precision; a chip that does not pay that penalty is built for scientific computing as well as for machine learning. On the AI side, the same silicon reaches 383 TFLOPS in the 16-bit formats that training commonly uses, and 383 TOPs in 4-bit and 8-bit integers, the low-precision formats used for inference.

Memory

The accelerator carries 128 GB of HBM2e memory on an 8,192-bit interface running at 1.6 GHz, which works out to 3.2 TB/s of bandwidth^[1]. High-bandwidth memory stacks DRAM dies vertically and links them to the GPU over a very wide bus, which is how a modest clock frequency yields terabytes per second. AMD's white paper also gives 6.96 TB/s of peak theoretical L2 cache slice bandwidth^[2] — the on-chip cache that sits between the compute units and the off-chip memory. Capacity matters as much as speed: 128 GB is where model parameters or simulation data have to fit, and bandwidth is what keeps the arithmetic lanes from idling.

Form factor and power

Thermal design power is 500 W or 560 W depending on configuration, and the product ships as a passive OAM module on PCIe 4.0 x16 with eight Infinity Fabric links at 100 GB/s each^[1]. Passive means the module has no fan of its own; the server chassis supplies the airflow. OAM (OCP Accelerator Module) is a mezzanine format — the accelerator is mounted on a baseboard inside the machine rather than plugged in like a graphics card — which makes it practical to pack several of them into one system. Infinity Fabric is AMD's interconnect, and eight links at 100 GB/s each give the accelerators a direct path for exchanging data with one another and with the host.

Where it sits

Read together, the specifications describe a chip asked to do two jobs at once: double-precision simulation on one side, low-precision neural networks on the other, with a large pool of fast memory shared between them. They also describe a part that cannot be used on its own. A power draw of 500 W or more, passive cooling, an OAM baseboard and high-speed fabric links all place the MI250X inside a complete server system rather than at the end of a cable, and that infrastructure is part of what the accelerator's numbers assume.