AWS Trainium2

What it is

AWS Trainium2 is the second generation of Trainium, the machine-learning chip Amazon Web Services designs for its own cloud. It succeeds the first-generation Trainium, and AWS puts the generational jump at up to four times the performance of the original chip.^[1] Trainium2 became generally available on 5 December 2024, when AWS announced it at its re:Invent conference.^[2] General availability is the point at which a cloud offering stops being a preview and can be bought and depended on by any customer.

How the chips are packaged

Customers do not buy Trainium2 as a standalone part; they rent it through EC2, AWS's compute service. AWS offers the chip in two shapes. A Trn2 instance contains up to 16 Trainium2 chips. A Trn2 UltraServer contains up to 64 chips, wired together with an interconnect AWS calls NeuronLink.^[1]

The second shape matters because of how large models are trained. A single chip holds only a fraction of a modern model, so the work is spread across many chips that must exchange data constantly; the links between chips, not just the chips themselves, set the pace. AWS quotes floating-point throughput for both shapes: up to 20.8 petaflops for a 16-chip Trn2 instance, and up to 83.2 petaflops for a 64-chip Trn2 UltraServer.^[2] A petaflop is a quadrillion floating-point operations per second, the usual shorthand for how much numerical work a system can do.

Performance and price

AWS makes its case in cost as well as speed. It says Trn2 instances and Trn2 UltraServers offer 30–40% better price performance than the GPU-based EC2 P5e and P5en instances.^[1] Price performance means the amount of compute delivered per unit of money spent. That the comparison is drawn against GPU instances indicates where AWS is placing the chip: as an alternative to general-purpose graphics processors for AI work. These are AWS's own figures, measured on yardsticks AWS defines.

Software

Hardware is only half of a chip's usefulness. AWS's software stack for its own silicon, the Neuron SDK, integrates natively with PyTorch and JAX and supports more than 100,000 models in the Hugging Face library.^[2] The practical point is that a model written against mainstream frameworks can be moved onto Trainium without being rewritten from scratch, which is what makes a non-GPU chip usable by teams that did not build their code around it.

Where it sits

The largest deployment AWS described is Project Rainier, which it said it was building with Anthropic: an EC2 UltraCluster using hundreds of thousands of Trainium2 chips, with more than five times the compute Anthropic used for its then-current leading model.^[2] That is a different order of scale from a single instance, and it shows the direction of the product line: from a chip that one team can rent by the instance, up to clusters of hundreds of thousands of chips serving a single model developer. For AWS, Trainium2 is thus less a component than a bid to run AI workloads on silicon it designs itself, from the interconnect upward.