AMD Instinct MI350X

What it is

The AMD Instinct MI350X is a data-center GPU for artificial intelligence and scientific computing, one of the accelerators in AMD's Instinct MI350 series. AMD says the series is built on the CDNA 4 architecture, the compute-oriented line of the company's GPU designs, and that each accelerator carries up to 288 GB of HBM3E memory ^[1].

AMD's specification lists 288 GB of HBM3E for the MI350X, supplied by Micron and Samsung Electronics, with up to 8 TB/s of bandwidth ^[1]. A Chinese-language specification table gives the typical board power as up to 1,000 W ^[2]. Capacity, bandwidth and power are the numbers that decide how such a card is used: capacity sets how large a model fits in memory, bandwidth sets how quickly the arithmetic units can be fed, and power sets what a rack of them costs to run and cool.

Background

AMD published the MI350 series announcement on 12 June 2025 ^[1]. The company presents the series as a generational step, claiming up to 4 times the generation-on-generation AI compute and up to a 35 times leap in inferencing performance ^[1]. These are AMD's own comparisons between the new series and its predecessor, not measurements against other products.

Specifications

Memory is the headline number. Each MI350X carries up to 288 GB of HBM3E, and an eight-GPU MI350X platform provides 2.3 TB of HBM3E in total ^[1]. HBM (high bandwidth memory) stacks memory dies vertically next to the GPU and connects them over a very wide interface, which is how one package reaches hundreds of gigabytes at terabyte-per-second bandwidth. Large models are usually split across several GPUs, and long inputs need extra memory for cached intermediate state, so both per-accelerator capacity and the total across a platform matter.

AMD's specification table lists 72.1 teraflops of FP64 and 2.3 petaflops of FP16 matrix performance, then 4.6 petaflops of MXFP8/OCP-FP8, 9.2 petaflops of MXFP6 and 9.2 petaflops of MXFP4 matrix performance ^[1]. Double precision (FP64) is the format used in numerical simulation, where rounding errors accumulate over long calculations; FP16 and the narrower 8-, 6- and 4-bit formats are used for neural networks, where lower precision buys more throughput per watt. The table shows the trade: halving the number of bits roughly doubles the quoted matrix throughput, which is why the 6-bit and 4-bit entries are the largest.

Racks and cooling

AMD says the MI350 series supports up to 64 GPUs in an air-cooled rack and up to 128 GPUs in a direct liquid-cooled rack, delivering up to 1.3 exaflops of MXFP4/MXFP6 performance ^[1]. An exaflop is a billion billion floating-point operations per second. The two rack designs are not decoration: with a board power of up to 1,000 W ^[2], a rack of accelerators produces heat that fans alone struggle to remove, so direct liquid cooling — coolant piped to a plate mounted on the chip — is offered as the denser option.

Where it fits

Data-center accelerators are no longer judged one card at a time. Buyers compare memory capacity, memory bandwidth, throughput in the low-precision formats that inference actually uses, and the power and cooling a full rack requires. The MI350X's published numbers — 288 GB of HBM3E at up to 8 TB/s, up to 1,000 W per board, and rack configurations running to 128 GPUs — describe a chip aimed at training and serving large models at the scale of a rack or a data-center hall rather than a single server.