NVIDIA unveils Volta and Tesla V100 at GTC 2017

A GPU aimed at a different workload

Graphics processors were originally designed to fill screens with pixels, an operation that splits naturally into thousands of small parallel calculations. Neural networks turn out to need the same kind of parallel arithmetic, and NVIDIA had been selling data center GPUs into deep learning and scientific computing. Volta was announced as the architecture created to drive the next wave of advancement in AI and high performance computing^[1].

The keynote

Founder and CEO Jensen Huang unveiled Volta in his GTC keynote on May 10, 2017^[1][2]. NVIDIA described it as the world's most powerful GPU computing architecture^[1]. At the same event the company announced the first processor built on that architecture, the NVIDIA Tesla V100 data center GPU^[1] — a part intended for servers rather than desktop machines.

Inside the V100

Volta is NVIDIA's seventh-generation GPU architecture and is built from 21 billion transistors^[1]. The V100 carries 640 Tensor Cores^[1], specialized circuits that perform the small matrix multiplications of which neural network training mostly consists, doing them directly in hardware rather than through general-purpose instructions. NVIDIA quotes 120 teraflops of deep learning performance for the chip, a figure it presented as equivalent to the performance of 100 CPUs^[1].

Memory is the other half of the design. The V100 has 900 GB/sec HBM2 DRAM, developed in collaboration with Samsung, giving 50 percent more memory bandwidth than previous generation GPUs^[1]. Bandwidth matters because training a network means moving weights and data between memory and processors constantly; if the memory cannot feed the arithmetic units, those units wait. NVIDIA also gave the generational comparison: a 5x improvement over Pascal in peak teraflops and 15x over the Maxwell architecture^[1].

A complete system, and its price

The chip was not the only thing announced with a number attached. A DGX-1 system equipped with eight Volta GPUs was priced at US$149,000^[3]. A DGX-1 is a single server holding eight GPUs, so that price describes a working machine rather than a component, and it sets the order of magnitude for what a group had to spend to get Volta-class compute in house.

What the announcement argued

The pitch was comparative. Describing 120 teraflops as the equivalent of 100 CPUs, and setting the chip against Pascal and Maxwell, was an argument that a job which previously needed a large cluster of conventional processors could now run on far fewer GPUs^[1]. Naming AI and high performance computing as the two target areas pointed the part at both the deep learning boom and traditional simulation work. Peak figures describe a ceiling rather than behaviour under a typical workload, which is worth keeping in mind when reading such comparisons.