Apart from exceptions such as intrinsically disordered proteins (IDPs), a folded protein’s amino acid sequence is typically assumed to fully determine its structure. Fold-switching proteins challenge the one-sequence-one-structure paradigm, suggesting that there is still much to be learned about protein folding and binding1. With myriad protein sequences and structures now available, exploring the protein universe is feasible to a large extent now. AlphaFold (AF)–a deep learning-based model has been immensely successful in predicting protein structure from sequence2. We were curious to explore AF’s capabilities on fold-switching proteins and to understand what drives its predictions.
Previously, we hypothesized that AF2 may be using sophisticated pattern recognition to search for the most probable conformer rather than protein biophysics to model a protein’s structural ensemble3, however, many AlphaFold users believe that AF2 predictions result from coevolutionary inference combined with some physical knowledge of protein folding4. Hence, we were eager to understand the mechanisms behind the tool and systematically benchmark all versions of AF and AF-based methods that employ enhanced sampling to get predictions of alternative conformations of fold-switching proteins.
Adequate sampling of distinct experimentally consistent conformations of the same protein.
We tested AF’s ability to predict two folds assumed by single sequences on 92 pairs of experimentally determined fold switchers from many diverse fold families and source organisms1. AF predictions were considered successful when they accurately reproduced both experimentally determined conformations (Fold1 and Fold2) of the fold-switching protein. Using several versions of AF (including AF3) with/without templates and two newly developed methods of enhanced sampling, we generated ~280,000 models. Anticlimactically, after sifting through all of them, we found that AF sampled both conformations in only 32/92 (35%) fold-switchers that were likely in its training set (Figure 1).
Figure 1. AF2 predicts fold switching with modest success. (a) Numbers of successful fold-switch predictions for each AF2 method and AF3 compared with coevolutionary information found for both folds (ACE) and the total number of possible successes (dotted red line). All_AF2 combines all unique successful predictions from all AF2-based methods: >282,000 predictions. Predictions successfully made by more than one AF method are black; predictions unique to each method are gray.
Notably, a recent computational approach called Alternative Contact Enhancement (ACE) developed in our lab identified coevolutionary information unique to both folds of 56-fold-switching proteins, confirming that MSAs often contain structural information unique to both conformations5. Hence, the enhanced sampling approaches we tested appear not to leverage the dual-fold coevolutionary information present in many MSAs of fold-switching proteins. Additionally, we found that for 30-49% of predictions generated from these enhanced sampling methods do not resemble either experimentally determined structure.
AF2 confidence metrics select against alternative conformations of fold switchers.
To describe how well AF2 could distinguish between good and inaccurate predictions, we assessed the relationship between prediction quality and AF2’s confidence metrics (pLDDT and pTM). Remarkably, we found both scores assigned lower confidences to diverse correctly predicted conformers and higher confidences to predictions that have not been observed experimentally! This suggests that AF2’s confidence metrics select against experimentally consistent predictions of fold switchers, especially Fold2, in favor of experimentally inconsistent predictions.
So far, we have established that AF modestly samples alternate conformation, often mis-predicting and assigning lower confidence to correctly detected alternate conformers. Taken together, this implies that AF is a weak predictor of fold switching. This brings us to the pivotal question we asked in the paper: what drives AF predictions and why does it fail on fold-switching? We found direct evidence indicating that AF2 has memorized structural information during training. Although the coevolutionary information from a protein called RfaH consistently corresponded to its b-sheet conformation during each step of prediction, AF2 ultimately predicted its helical conformation instead (Figure 2). Furthermore, AF3 misassigned coevolutionary restraints, as seen by incorrect prediction of dimeric XCL1 while AF2 predicted it correctly by structure memorization.
Leveraging “structure memorization” and hope for the future
These findings have important implications. Without coevolutionary information or full knowledge of protein folding physics, AlphaFold will likely be constrained to sample alternative conformations learned during training, limiting the sorts of new folds and fold switchers it can probably discover. However, we found that random sequence sampling may enable robust predictions of some fold-switching proteins6, suggesting that memorization and sequences association in AF2 may be used for robust sampling of alternate predictions in some cases.
Figure 2. AF2 structure predictions can be inconsistent with structural restraints from Evoformer. Although the full AF2 model predicts the autoinhibited form of RfaH (green helical structure, left panel) after 2 recycles (R2), the evolutionary restraints from Evoformer correspond to its active β-sheet form (blue β-sheet structures, right panel and Fig.S12) from each MSA inputted into the full AF2 model (left panel). The initial input MSA is depicted in the top lefthand corner with target sequence bold and colored black, blue, and yellow. Randomly subsampled MSAs inputted at each recycle are depicted in both panels, with identical MSAs being inputted at R0,1,2 and MSA_R0.0, MSA_R1.0, MSA_R2.0, respectively. The right and left panels differ by how AF2 makes predictions. In the right panel, restraints from input MSAs should inform the predictions because the input MSA is passed through AF2 (Evoformer and Structure Module) only once (0 recycles); this also applies to the R0 (0 recycles) step in the left panel. All structures based on these MSA restraints output structures with β-sheet CTDs (blue). The recycling steps in the left panel (R1 and R2) differ because they update the prediction with both previous MSA restraints and the previously predicted structures from the Structure Module. In these cases, the CTD becomes increasingly helical (green regions), indicating that the prediction changes during the recycling process. Right and left panels are shaded to represent what information drives predictions: beige (recycling process, left) and light blue (Evoformer, right).
Reference:
Porter, L. L. & Looger, L. L. Extant fold-switching proteins are widespread. Proc Natl Acad Sci U S A 115, 5968-5973, doi:10.1073/pnas.1800168115, (2018).
Jumper, J. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583-589, doi:10.1038/s41586-021-03819-2, (2021).
Chakravarty, D. & Porter, L. L. AlphaFold2 fails to predict protein fold switching. Protein Sci 31, e4353, doi:10.1002/pro.4353, (2022).
Sala, D., et al. Modeling conformational states of proteins with AlphaFold. Curr Opin Struct. Biol.81: 102645, https://doi.org/10.1016/j.sbi.2023.102645, (2023).
Schafer, J. W. & Porter, L. Evolutionary selection of proteins with two folds. Nat Commun 14, 5478, https://doi.org/10.1038/s41467-023-41237-2, (2023).
Schafer, J. W., Chakravarty, D., Chen, E. A. & Porter, L. L. Sequence clustering confounds AlphaFold2. bioRxiv, 2024.2001. 2005.574434 (2024).