NVIDIA DGX-1

A machine, not a chip

The DGX-1 is a computer that NVIDIA unveiled at its GPU Technology Conference in April 2016 and billed as the world's first deep learning supercomputer^[1]. It is not a processor but a complete system: eight Tesla P100 GPUs with 16 GB of memory each, plus the CPUs, memory, storage and power delivery needed to keep them fed, all inside one chassis^[1][2]. A customer bought a finished unit rather than assembling a cluster from parts.

Why eight GPUs in one box

Training a neural network is a loop: show the model a batch of examples, measure how wrong its answers are, then nudge every parameter a little in the direction that reduces the error. The arithmetic is dominated by matrix multiplication, which splits cleanly into thousands of independent operations — the kind of work a GPU, with its many parallel cores, does well. Putting eight of them in one machine matters because they can be connected to each other more tightly than eight separate servers can, and because the parameters of a large model have to fit somewhere. NVIDIA's headline claim was that one DGX-1 delivered the throughput of 250 x86 servers^[1]. That is a vendor's comparison, not an independent measurement, but it captures the argument the machine was making: a dense box of accelerators stands in for a room of general-purpose servers.

What was in the box

NVIDIA configured the system with eight Tesla P100 GPUs of 16 GB each and rated it at up to 170 teraflops of half-precision peak performance^[1][2]. The published datasheet lists dual 20-core Intel Xeon E5-2698 v4 CPUs, 512 GB of system memory and 7 TB of SSD storage, in a 3U chassis with a maximum power draw of 3,200 W^[1][2]. Those numbers describe data-centre equipment rather than a workstation sitting under a desk: it needs substantial power and cooling around it.

What half precision means

The 170 teraflops figure is a peak for half precision, the format that stores each floating-point number in 16 bits instead of the 32 used by ordinary single precision. Fewer bits mean less data to move and more arithmetic per second, at the cost of narrower range and coarser accuracy. Deep learning tolerates that looseness better than most numerical work does, which is why a half-precision number was the one NVIDIA put at the front of the specification. Peak figures are ceilings: a real training run reaches a fraction of them, and the number says nothing on its own about how quickly a particular model learns.

Where it sits

Before machines of this kind, deep learning groups mostly bought accelerator cards and built their own servers around them. The DGX-1 sold the whole assembly — GPUs, CPUs, storage, power and cooling — as one product with one set of numbers on the sheet^[1]. That packaging is the part that mattered: the number of accelerators became a single quantity a lab could order, and the machine gave the field a reference configuration to compare against.