Triton

What Triton is

Triton is an open-source programming language and compiler for writing GPU kernels — the small programs that run in parallel on a graphics processor. Matrix multiplication, normalization and the other heavy operations inside neural networks all end up as kernels. Triton's syntax is close to Python's, and OpenAI released version 1.0 in July 2021 with a plain goal: let researchers with no CUDA experience write highly efficient GPU code, most of the time on par with what an expert would produce^[1].

Where it came from

Triton was not written from scratch inside OpenAI. The company describes its work as extending and improving Triton, a recent language and compiler whose original creator now works at OpenAI^[1]. The project is developed in the open: the triton-lang/triton repository is released under the MIT license and has roughly 20,000 stars, and it was later moved from the openai organization to the separate triton-lang organization^[2].

How it works

In Triton a kernel is an ordinary Python function with a decorator, and it is compiled just in time rather than ahead of time^[1]. The real design choice is the level at which parallelism is expressed. CUDA programmers work with the SIMT (single instruction, multiple threads) execution model, tracking what each individual thread does. Triton instead exposes parallelism inside a processing instance through operations on blocks^[1].

The compiler then takes over the bookkeeping that normally separates a working kernel from a fast one. It automates memory coalescing, shared memory management and scheduling within streaming multiprocessors, the GPU's individual compute units; only scheduling across those units is left to the developer^[1]. Memory coalescing is worth spelling out: when neighbouring threads read neighbouring addresses, a single memory transaction is used fully instead of wasting bandwidth. Details like this used to be a mark of GPU experience, and Triton moves them into the compiler.

What it achieves

OpenAI reported two concrete results. First, Triton can express FP16 matrix multiplication kernels that match the performance of cuBLAS — a widely used GPU library for such operations — in under 25 lines of code, something many GPU programmers cannot do by hand^[1]. Second, OpenAI researchers had already used Triton to produce kernels up to 2x more efficient than equivalent Torch implementations^[1].

Where it sits

Triton sits between two extremes of GPU programming: hand-written CUDA, which gives full control at the cost of a long learning curve, and high-level frameworks that generate kernels the user never sees. Its pitch is that a researcher can keep writing something that looks like Python, describe work in blocks, and still get near-expert speed. The boundary it draws is deliberate — intra-processor scheduling is automated, cross-processor scheduling stays with the developer — so it is a tool for people who want to shape a kernel, not a system that hides the hardware completely. Combined with an MIT license and a repository that later moved out of OpenAI's own GitHub organization, that made it a piece of infrastructure others could build on rather than a feature locked inside one company's stack^[2].