AMD Instinct MI355X

What the MI355X is

The AMD Instinct MI355X is a data-center graphics processor (GPU) built for artificial-intelligence workloads. It belongs to AMD's Instinct MI350 series, which the company introduced on 12 June 2025.^[1] The chip is based on CDNA 4, the architecture AMD uses for its compute-oriented Instinct accelerators rather than for graphics cards whose job is drawing images.^[1]

Accelerators of this kind are not desktop parts. They are mounted several to a server, and the server is what a cloud provider or a research lab actually buys. Most of the numbers below therefore describe a single chip, while the most useful number for a buyer — how much memory and how much throughput a whole machine has — is the one given for an eight-GPU platform.

Memory and arithmetic

AMD's specification table lists 288 GB of HBM3E memory and 8 TB/s of memory bandwidth for the MI355X.^[1] HBM (high-bandwidth memory) is made by stacking memory dies vertically and placing them on the same package as the processor, which shortens the distance data travels between compute and memory and allows very high transfer rates. Capacity matters because the weights of a large model, plus the intermediate values that accumulate while the model answers a long prompt, all have to fit in memory; when they do not fit, work has to be split across chips or moved back and forth over slower links.

On the compute side, AMD lists up to 256 compute units per chip.^[1] A compute unit is one of the repeated processing blocks that a GPU switches between to keep its arithmetic pipelines busy; more of them, at a given clock speed, means more parallel work in flight.

The published precision table gives 78.6 teraflops of FP64 and 2.5 petaflops of FP16 matrix performance.^[1] FP64, double precision, is the format still used in parts of scientific simulation; FP16, half precision, is the format most machine-learning work moved to years ago. The table also lists 5 petaflops of MXFP8/OCP-FP8, 10.1 petaflops of MXFP6 and 10.1 petaflops of MXFP4 matrix performance.^[1] The MX formats are microscaling formats: values are stored in small blocks that share one scale factor, so a block can use short 8-, 6- or 4-bit numbers without losing too much range. Cutting the bits per value roughly doubles the number of values a chip can process in the same time, which is why the MXFP8 figure is half the MXFP4 figure. The pattern in the table — a comparatively small FP64 number next to very large low-precision numbers — reflects a chip shaped around inference and training of neural networks rather than around traditional double-precision simulation.

Whole systems, power and cooling

AMD says an eight-GPU MI355X platform provides 2.3 TB of HBM3E and 80.5 petaflops of MXFP4 performance.^[1] Those are the figures that describe a complete server node, and they are the ones that determine which models fit on one machine.

AMD's Chinese-language specification table lists a typical board power of up to 1,400 W for the MI355X and says it uses direct liquid cooling.^[2] Direct liquid cooling means coolant is piped to a cold plate attached to the chip instead of relying on air pushed through the chassis by fans. At board powers in the four-digit-watt range, air alone becomes impractical, so liquid cooling is a design requirement rather than an option.

Software and availability

AMD launched ROCm 7 alongside the MI350 series.^[1] ROCm is AMD's software platform for GPU computing: the layer that compilers, numerical libraries and machine-learning frameworks build on. A chip is only usable in practice if the software stack supports the operations a model needs, so a new release of the stack is part of the product launch, not an accessory to it.

AMD says the MI350 series is available through major cloud providers and is integrated into systems by Dell, HPE and Supermicro.^[1] Cloud availability matters because many teams never buy accelerators directly; they rent them by the hour.

Where it sits

AMD positioned the MI350 series against Nvidia's Blackwell generation for generative AI inference, and described a 35-fold inference improvement over its own CDNA 3 accelerators.^[1] Inference is the act of running an already-trained model to produce answers, as distinct from training, and it is the workload that dominates the cost of serving a chatbot. Comparisons of this kind depend on which models, batch sizes, precisions and software versions were used, so the headline multiple describes a particular measurement rather than a universal ratio.