Durk Kingma

Overview

Durk Kingma is a machine learning researcher whose name is attached to two widely used pieces of the deep learning toolkit: the variational autoencoder (VAE), a generative model, and Adam, an optimization method for training neural networks. His career has run through several of the field's main laboratories — Yann LeCun's lab at New York University, OpenAI, Google Brain / DeepMind and Anthropic ^[1].

Education and early research

Kingma earned a PhD (cum laude) from the University of Amsterdam between 2013 and 2017. His advisor was Max Welling, and his thesis was titled "Variational Inference and Deep Learning: A New Synthesis" ^[1]. He also spent time as a junior research scientist in Yann LeCun's lab at New York University, listed on his own site for 2009 and 2012 ^[1].

The variational autoencoder

Generative models try to learn enough about the structure of data that new examples can be sampled from the learned distribution. One route is to introduce a small set of unobserved variables — a latent representation — and learn a mapping from those variables to data, plus a mapping back from data to variables. Done naively, this runs into integrals that cannot be computed exactly. The paper "Auto-Encoding Variational Bayes", written with Max Welling and submitted to arXiv on 20 December 2013, introduces a way to train such models by optimizing a tractable lower bound on the data likelihood, with neural networks used for both directions of the mapping ^[3]. Kingma's site lists the variational autoencoder among his contributions ^[1].

Adam

Training a neural network means adjusting its parameters to reduce an error measure, usually by following the gradient of a loss function. Plain stochastic gradient descent applies one global step size to everything. "Adam: A Method for Stochastic Optimization", written with Jimmy Ba and submitted to arXiv on 22 December 2014, instead keeps running estimates of the first and second moments of the gradients and uses them to set a separate effective step size for each parameter ^[2]. Because the step size adapts per parameter, the same optimizer can be applied to problems whose parameters have very different scales, and training is less sensitive to the choice of learning rate.

Industry positions

Kingma was part of the founding team at OpenAI in San Francisco, serving as a research scientist from 2015 to 2018 and leading the Algorithms team ^[1]. From 2018 to 2024 he was a research scientist at Google Brain / DeepMind, where he led projects on generative models for text, image and video ^[1]. Since 2024 he has been researching large-scale machine learning at Anthropic ^[1].

Other work and recognition

His site lists Glow and Variational Diffusion Models among his contributions, alongside the VAE and Adam ^[1]. It also lists several awards: the 2015 Google European Doctoral Fellowship in Deep Learning, the 2019 ELLIS PhD Award and the 2019 Dutch Datascience Award ^[1]. The VAE paper received the ICLR 2024 Test of Time Award, and the Adam paper, jointly with Jimmy Ba, received the ICLR 2025 Test of Time Award ^[1]. The two threads those awards recognise — generative models that produce text, images and video, and the optimization machinery that makes large-scale training work — run through the positions he has held since.