NVIDIA L40S

What the L40S is

The NVIDIA L40S is a data-centre processor built on Nvidia's Ada Lovelace architecture. Nvidia positions it for both AI and graphics workloads in data centres, rather than for either one alone^[1]. That double role explains the shape of the specification: a large memory pool, high matrix-multiply throughput for neural networks, and a full set of ray-tracing and video engines for rendering and media work.

Memory and arithmetic

The card carries 48 GB of GDDR6 memory with error-correcting code (ECC) and 864 GB/s of memory bandwidth^[1]. ECC lets the memory detect and correct the occasional flipped bit, which matters in machines that run for weeks on long jobs. Memory capacity sets how large a model, batch or scene can be held on the card at once, while bandwidth sets how quickly that data can be fed to the compute units.

For arithmetic, Nvidia quotes 91.6 TFLOPS of FP32 throughput, 366 TFLOPS at TF32 on the Tensor Cores, 733 TFLOPS at FP16 and 1,466 TFLOPS at FP8; the last three figures are quoted with sparsity^[1]. Sparsity refers to networks in which many weights have been pruned to zero, so part of the multiply work can be skipped; a sparse figure therefore describes a best case rather than what an ordinary dense model will see. Lower-precision formats such as FP16 and FP8 trade range or accuracy for throughput, which is why they are widely used when running trained networks rather than training from scratch.

The chip contains 18,176 CUDA cores, 568 fourth-generation Tensor Cores and 142 third-generation RT Cores, with RT Core throughput stated as 212 TFLOPS^[1]. Lenovo's specification sheet for the same part gives 209 TFLOPS for the RT Cores^[2], a small gap that shows how vendor numbers can be rounded or measured under slightly different conditions.

Shape, power and host connection

The L40S is a dual-slot, full-height card measuring 4.4 inches by 10.5 inches, fitted with a 16-pin power connector and passive cooling^[1]. Passive cooling means there is no fan on the board itself: the server chassis pushes air through the card's heatsink, so the card belongs in machines whose airflow was laid out for it. Maximum power consumption is 350 W, and the card connects to the host over PCIe Gen4 x16 at 64 GB/s bidirectional^[1].

What it leaves out

The card supports neither NVLink nor Multi-Instance GPU partitioning^[1][2]. NVLink is Nvidia's direct high-bandwidth link between GPUs; without it, cards exchange data over PCIe or the network instead. MIG carves a single GPU into isolated instances, each with its own share of memory; without it, a card is shared between users through software such as vGPU rather than by hard partitioning. The absence of both features says something about the deployments the card assumes: many largely independent jobs in ordinary servers, rather than one tightly coupled job spread across a fabric of linked GPUs.

Media engines and virtualisation

The card includes three NVENC encoders and three NVDEC decoders, with AV1 encode and decode^[1]. AV1 is a video compression format, and dedicated engines handle it without tying up general-purpose compute cores. The card also supports vGPU software^[1], so a single physical card can be divided among virtual machines.

How to read the numbers

Taken together, the specification describes a card that puts a large ECC memory pool and two kinds of compute — Tensor Cores for neural networks, RT Cores for ray-traced graphics — onto a standard passively cooled PCIe board, and pays for it by giving up multi-GPU direct interconnect and hardware partitioning. The TFLOPS figures should be read with their footnotes: the Tensor Core numbers assume sparsity, the RT Core number differs slightly between Nvidia's and Lenovo's sheets, and none of them says on its own how fast a particular workload will run. What the sheet does establish clearly is the memory size, the power envelope, the host interface and the features that are not there — usually the details that decide whether a card fits a given server and a given job.