NVIDIA DGX-2

A server built around a number

NVIDIA announced the DGX-2 at its GTC conference in March 2018 and described it as the first single server to deliver two petaflops of computing power ^[1]. A petaflop is a measure of throughput: one quadrillion (10^15) floating-point operations per second. The figure matters because training a large neural network is mostly a matter of doing enormous numbers of multiplications and additions, and the faster those can be done, the less time a training run takes.

The DGX-2 is not a chip but a complete server — a chassis holding many processors, their memory, and the wiring between them, sold as a single unit that a lab or data centre can plug in and use.

What is inside

According to NVIDIA's datasheet, the system carries 16 Tesla V100 GPUs with 512 GB of total GPU memory, alongside 12 NVSwitches, 81,920 CUDA cores and 1.5 TB of system memory ^[2]. CUDA cores are the individual arithmetic units inside an NVIDIA GPU that execute the parallel threads of a program.

Maximum power usage is listed at 10 kW ^[2]. That number is worth pausing on: 10 kW is the kind of draw associated with a small cluster rather than a workstation, and it shapes where a machine like this can be installed and how it must be cooled.

NVSwitch, introduced with the machine

NVIDIA brought out NVSwitch together with the DGX-2. It connects all 16 GPUs at 2.4 TB/s of aggregate bandwidth ^[1][2].

The problem a switch solves is one of topology. Inside a server, GPUs need to exchange data with each other, and the cost of that exchange depends on how the links are arranged. A switch acts as a central point that many GPUs can connect through, so that traffic between any pair of them does not have to be squeezed through a fixed partner link. With 16 GPUs in one box, that difference determines whether the machine behaves like one large accelerator or like sixteen separate ones.

Why the comparison to a data centre

NVIDIA framed the DGX-2 in terms of the equipment it replaces, saying it had the deep learning processing power of 300 servers occupying 15 racks while being 60 times smaller ^[1]. The claim is a packaging argument as much as a performance one: the same amount of computing squeezed into a fraction of the floor space, with the interconnect to keep the parts working together.

That framing points at what such a system is for. Work that previously had to be split across many machines, with the network between them becoming the bottleneck, can in principle be held inside one chassis. For researchers, it changes the unit of purchase from a cluster to an appliance.

Where it sits

The DGX-2 is an expression of a particular bet: that the fastest path to better AI results runs through very large, very tightly coupled GPU systems, and that customers will buy them as integrated products rather than assembling them. The two-petaflop figure, the 16 GPUs and the 10 kW power envelope together define the shape of that bet — a lot of computing in one place, at a density that makes power and cooling part of the design conversation.