John Schulman

Overview

John Schulman is an American artificial intelligence researcher, born in 1987 or 1988. He was a member of the US Physics Olympiad team in 2005.^[1] His work centres on reinforcement learning — the branch of machine learning in which a system improves by trial and error, adjusting its behaviour toward whatever earns a reward — and on applying that machinery to the training of large language models.

Education

Schulman took a physics degree at Caltech, completing it in 2010, and then moved into computer science. He earned a PhD in electrical engineering and computer sciences at UC Berkeley, where his adviser was Pieter Abbeel.^[1] The move from physics to machine learning is a common path in the field: physics training emphasises mathematical modelling and careful reasoning about systems, which transfers readily to problems such as how an agent should choose actions in an uncertain environment.

Main contributions

Schulman's best-known technical work is a pair of policy-optimisation algorithms. Trust Region Policy Optimization, published in 2015, and its successor Proximal Policy Optimization, published in 2017, address a practical difficulty in reinforcement learning: when a learning agent updates its policy — the rule that maps situations to actions — a large update can destroy progress made so far, while a very small one makes learning crawl. These methods constrain each update to a limited region, so the policy improves steadily rather than lurching. Proximal Policy Optimization simplified the earlier method's machinery into a clipped objective that is easier to implement and tune. Both algorithms became components of ChatGPT's training.^[1]

He is also a co-author of the InstructGPT paper, published in 2022, on training language models to follow instructions with human feedback.^[2] The approach described there, often called reinforcement learning from human feedback, works in stages: people compare or rank model outputs, a reward model is trained on those preferences, and the language model is then optimised against the reward model. The result is a model that produces answers people rate as more useful, rather than merely plausible continuations of text.

At OpenAI

Schulman co-founded OpenAI in December 2015, shortly before finishing his PhD.^[1] There he led the company's reinforcement-learning team, which created ChatGPT.^[1] He has been called the "architect" of ChatGPT, a description that points to his role in the training pipeline rather than in any single product decision.^[1]

Later moves

In August 2024 he left OpenAI for Anthropic, saying he wanted to focus more on AI alignment and hands-on technical work.^[1] The move was notable because it was a step toward alignment research — work on making AI systems behave as their designers intend — at a moment when many researchers were weighing how much of their time to spend on capability versus safety questions.

His stay at Anthropic was short. In February 2025 he left to become chief scientist of Thinking Machines Lab.^[1]

Why it matters

The methods Schulman worked on became part of the standard toolkit for turning a raw language model into a usable assistant. Proximal Policy Optimization in particular is widely used in reinforcement learning generally, not only for language models, because it is comparatively simple and robust. The InstructGPT-style recipe — collecting human preferences, fitting a reward model, then optimising the model against it — became a common way to make models follow instructions. Schulman's career also illustrates a pattern in the field: researchers who build the training methods underneath a widely used product are often less visible than the product itself, even when their algorithms are the reason it works.