xAI brings its Colossus supercomputer online in Memphis

Background

xAI, the AI company behind the Grok models, needed large amounts of computing power to train them. Colossus was the cluster it built for that purpose, in partnership with NVIDIA.

What happened

xAI's Colossus cluster, which contained about 100,000 NVIDIA H100 GPUs, went into operation between July and September 2024. It was housed in a former Electrolux plant in South Memphis, Tennessee^[2][1].

xAI and NVIDIA built the facility and system in 122 days. Training began about 19 days after the first rack arrived^[1].

Key details

Colossus uses NVIDIA's Spectrum-X Ethernet networking, with Spectrum SN5600 800Gb/s switches and BlueField-3 SuperNICs. NVIDIA reported 95% data throughput for the system^[1].

Musk called it "the most powerful training system in the world". In October 2024 NVIDIA said xAI was doubling the system to 200,000 Hopper GPUs^[1].

Colossus was used mainly to train Grok. Grok 3 was trained on the Colossus supercluster, and Grok 4's reinforcement learning ran on the 200,000-GPU version^[2][3][4].

Power came at first from a grid connection of about 8 MW. It was supplemented by temporary natural-gas turbines and Tesla Megapack batteries. In April 2026 the NAACP, represented by the Southern Environmental Law Center and Earthjustice, sued over compliance with the Clean Air Act^[2].

A second site, Colossus 2 in Whitehaven, Memphis, brought its first major cluster online around January 2026. The cluster had about 110,000 GB200-class processors and roughly 210 MW of power. By late September 2026, 440,000 GB300 GPUs were in use there^[2].

On 30 December 2025 xAI bought a third Memphis building. The aim was nearly 2 GW of compute and at least 1 million GPUs^[5].

Impact

Colossus was the system on which Grok 3 and Grok 4 were trained^[2][3][4]. The Memphis operation later grew to a second site and a third building^[2][5].

For readers, a GPU (graphics processing unit) is a chip originally built for graphics that became the main hardware for training large AI models. A cluster is a group of linked computers that work together on one job. A training run adjusts a model's parameters using large amounts of data, and a cluster of this size can run many such jobs over the course of a model's development^[1][2].