NVIDIA A10

What it is

The NVIDIA A10 is a GPU for data-center servers. NVIDIA built it on its Ampere architecture, and it carries second-generation RT Cores and third-generation Tensor Cores^[1]. Those two blocks are what let one card do two kinds of work: the RT Cores handle ray tracing, the geometry-heavy math behind realistic rendering, while the Tensor Cores handle the matrix arithmetic that neural networks are made of. Putting both on the same chip is the design idea behind the A10.

What is on the card

NVIDIA's datasheet lists 24 GB of GDDR6 memory and 600 GB/s of memory bandwidth, all inside a 150 W power envelope^[1]. Memory capacity and bandwidth matter for GPU work because large models and large images have to be moved in and out of memory constantly; 24 GB sets how big a model or scene can be held at once without splitting it up.

For compute, NVIDIA lists 31.2 TFLOPS of FP32, 125 TFLOPS of FP16 Tensor Core throughput and 250 INT8 TOPS, with sparsity doubling the Tensor Core figures^[1][2]. These numbers describe different precisions. FP32 is standard single-precision arithmetic. FP16 is a narrower format that stores numbers in half the bits, which is usually enough for neural network inference and lets the hardware do more operations per second. INT8 goes narrower still, using 8-bit integers, and is common for running already-trained models where a small loss of accuracy is acceptable. Sparsity refers to skipping weights that are zero, so the Tensor Core figures can double when a model has that structure.

Form factor

NVIDIA packaged the A10 as a single-slot, full-height, full-length PCIe Gen4 card^[1]. The single-slot width is a practical detail: a standard rack server has a fixed number of expansion slots, so a card that takes one slot rather than two lets a system hold more GPUs, or hold GPUs alongside other cards. PCIe Gen4 is the host interface the card plugs into.

What it is for

NVIDIA positions the A10 for graphics, virtual workstations, video and AI inference in mainstream enterprise servers^[1]. That is a deliberately broad list. A virtual workstation means a desktop running on a server and streamed to a remote user, which needs graphics acceleration in the data center. Video means encoding and decoding streams. AI inference means running a trained model to produce answers, as opposed to training it. The common thread is that all of these are workloads an ordinary company server might be asked to handle, rather than a dedicated training machine.

Where it sits

NVIDIA states that the A10 delivers up to 2.5 times the inference performance of the T4^[1], the earlier card it is meant to succeed in this role. The comparison is stated as an inference figure, which fits the positioning: the A10 is presented as a general-purpose accelerator for servers that have to serve several kinds of work at once, not as a card built for a single task.