EvolutionaryScale releases ESM3 and the esmGFP result

Background

Proteins do most of the chemical work in living cells, and what a protein does follows from the order of its amino acids and the three-dimensional shape that order folds into. Designing a sequence that folds as intended is hard, because the number of possible sequences is far larger than anything a laboratory could test. A protein language model borrows the approach of a text language model: trained on very many sequences, it learns which combinations of residues look plausible, and can then propose new sequences rather than only score existing ones.

What was released

On 25 June 2024 the company EvolutionaryScale published ESM3 together with a paper describing the esmGFP generation experiment ^[1]. The company said the model had been trained on one of the highest-throughput GPU clusters in the world at the time ^[1]. An open checkpoint was released alongside it.

Key details

The company reported that capability on hard design tasks such as atomic coordination emerges with scale and improves further after an alignment step ^[1]. Atomic coordination is the problem of placing particular atoms at the right distances and angles from one another; it is not something a model can get right by reproducing sequence statistics alone.

The alignment step uses feedback that the model generates on its own outputs, in a manner similar to reinforcement learning from human feedback ^[1]. In reinforcement learning from human feedback, a model's answers are ranked or scored and the model is adjusted toward the preferred ones; here that kind of adjustment is driven by feedback the model produces itself.

The release also covered a prompt-driven design of a scaffold for the active site of PETase, an enzyme that breaks down PET plastic ^[1]. An active site is the small set of residues where the reaction takes place, and the scaffold is the rest of the structure that holds those residues in the right geometry. Designing a scaffold around a given active site is a tightly constrained problem: the parts the designer cares about are fixed, and the remainder has to be built to support them.

The open checkpoint has its own timeline: its Hugging Face repository was created on 3 June 2024, ahead of the announcement ^[2]. The model card documents 2.78 billion natural protein sequences behind the open checkpoint ^[2], which gives a sense of the scale of sequence data involved.

What it means

The release is a data point on how protein design is being approached with general-purpose sequence models rather than only specialised, physics-based search. Two parts of the reported results are worth separating. One is scale: the claim is that a capability needed for precise design appears only once the model and its training are large enough, and can then be pushed further by alignment ^[1]. The other is a change in interface: instead of specifying a structure and searching for a sequence that fits it, a user can describe the design problem in a prompt, as in the PETase scaffold case ^[1].

The open checkpoint matters for a separate reason. By placing a checkpoint on Hugging Face and documenting the 2.78 billion natural sequences behind it ^[2], the company made a large protein model available for outside use rather than keeping it behind its own interface only. Timing was part of the release as well: the repository was created on 3 June 2024, before the 25 June 2024 announcement ^[1][2].