AlphaFold tops the CASP13 protein-structure assessment

Background

CASP — the Critical Assessment of Techniques for Protein Structure Prediction — is a blind assessment held every two years, in which research groups predict protein structures and are then scored against experimental structures that were not public when the predictions were made^[1]. The thirteenth edition, CASP13, was held on 1–4 December 2018^[3]. Proteins are chains of amino acids that fold into three-dimensional shapes, and a protein's function is closely tied to that shape, so inferring the shape from the sequence alone is a central problem in biology.

What happened

At CASP13, DeepMind's AlphaFold placed first overall^[1][2][3]. DeepMind said that AlphaFold achieved the highest accuracy among the participants^[4]. Because the assessment is blind, teams submit their predictions without seeing the experimental structures used for scoring, so the ranking reflects how well a method handles proteins whose structures were unknown at the time.

Key details

The margin showed most clearly on the hardest targets. Of the 43 hardest targets in the assessment, AlphaFold gave the best prediction for 25^[1]. Its median GDT score was 58.9, ahead of 52.5 and 52.4 for the next two teams^[1]. GDT measures how closely a predicted structure agrees with the experimental structure.

AlphaFold did not output only a single set of coordinates: it predicted distributions of distances between residues^[1][3]. That is, for a pair of residues it produced a probability distribution over how far apart they are, rather than one fixed distance. Such a representation keeps information about how uncertain each part of the prediction is, instead of committing to a single value.

Impact

Two years later, DeepMind's successor system AlphaFold2 reached a median GDT of 92.4 at CASP14^[1][4]. Set against the 58.9 median that AlphaFold recorded at CASP13^[1], the two results make the change in prediction accuracy between the two assessments easy to see, and they mark the point from which later work on the system is usually measured.