‘It had gotten to about 30 minutes before and I turned to my wife, and I said “Well, I guess it’s not this year.” … Then 30 seconds later, I got this phone call from Sweden.’
Waiting for the announcement of this year’s Nobel prize in chemistry, John Jumper thought he had a ‘one in 10 chance’ of winning. Many other observers would have put his chances much higher, such has been the impact of the AlphaFold platform Jumper and his colleagues at Google DeepMind created. AlphaFold’s ability to accurately predict proteins’ 3D structures has been repeatedly described as a revolution for biochemistry, a sentiment echoed by the Nobel committee as it announced that Jumper and DeepMind’s chief executive Demis Hassabis had scooped one half of this year’s prize.
The other half of the prize went to David Baker, a pioneer of protein design, whose lab at the University of Washington, US, has opened the door to an entirely new realm of unnatural proteins.
The tools developed by the DeepMind team and Baker’s lab use AI to tackle two related problems that have captivated scientists for decades: how to determine a protein’s structure given only the sequence of amino acids that is made from, and how to create new proteins that take on shapes and perform functions never before seen in nature.
A decades old problem
Every protein is formed of a chain of amino acids that folds itself into a unique twisted and tangled structure. This shape is integral to the protein’s biological activity, dictating how it will interact with other biomolecules with which it comes into contact. Common features include coils known as alpha-helices and sections where strands snake alongside each other forming larger units called beta-sheets.
In real life, proteins fold into their favoured configuration within seconds
In the mid-20th century Christian Anfinsen, a chemical biologist based mainly at the National Institutes for Health in Maryland, US, carried out a series of groundbreaking experiments using a protein called bovine pancreatic ribonuclease. He showed that if the ribonuclease was made to unfurl from its natural shape and then allowed to fold up again, it would always return to its original shape. This led Anfinsen to conclude that the protein’s native state was dictated by the sequence of amino acids from which it was formed – a finding that earned him a share of the 1972 chemistry Nobel prize.
Soon after this discovery, the molecular biologist Cyrus Levinthal from the Massachusetts Institute of Technology in the US observed that even a relatively short protein could potentially take on an unimaginably large number of different configurations. If the unfolded protein was to randomly sample trillions of possible states per second, it would still take eons to try them all. But in real life, proteins fold into their favoured configuration within seconds.
This led Levinthal to conclude that protein folding is ‘speeded and guided’ by local interactions along the amino acid chain that nucleate the process.
Anfinsen and Levinthal’s findings sparked one of biology’s grand challenges: if all the information needed to direct protein folding is stored in the amino acid sequence, then surely there must be a way to decode this. Doing so would enable scientists to predict a protein’s favoured shape, offering new ways to study how proteins work and accelerating scientific breakthroughs. Researchers could be freed from the painstaking work needed to coax proteins into forming crystals and characterise them through diffraction experiments.
The world cup of structure prediction
‘A lot of people interested in calculating things caught on to the idea,’ says John Moult, a computational biologist based at the University of Maryland, US. ‘And from around 1970 on, a whole industry emerged for doing it.’
I was shocked how hard it is to know whether something works or not
But as these efforts began, Moult explains that there wasn’t really a good way to evaluate the different methods. ‘I was shocked when I got into the calculation game how hard it is to know whether something works or it doesn’t. And it was clear that we weren’t looking rigorously enough at how we were doing, not properly assessing: OK, I tried this hypothesis – did it work or didn’t it?’
In 1994, Moult and his colleague Krzysztof Fidelis launched a new initiative called the Critical Assessment of protein Structure Prediction, or Casp, experiments. The concept is essentially a biennial ‘world cup’ for protein structure prediction: the best teams use their methods to predict the structure of a selection of proteins, and are then scored by comparing their results to unpublished crystallographic data.
Baker’s lab first entered Casp in its third iteration in 1998. His group’s method was based on analysing short protein sequences less than 10 amino acids residues in length and using this to model common local interactions that can arise in proteins. They also took into account longer distance interactions like main chain hydrogen bonding and the tendency for hydrophobic sidechains to position themselves within a protein’s core. The program was dubbed Rosetta after the stone that enabled Egyptian hieroglyphs to be decoded.
The Rosetta program would be continually developed and improved, becoming one of the top performing packages in all of the future Casp competitions.
Proteins by design
After making its debut at Casp3, Baker’s lab continued working on protein structure prediction, but also started a new strand of research that turned the problem on its head: if you wanted a protein to take on a certain shape, what sequence of amino acids would be required?
‘When I started here, I was really focused on trying to understand the principles of protein folding, and then to use those to develop models for the folding process and to predict protein structure,’ recalls Baker. ‘So we really only started working on protein design around the late 1990s when Brian Kuhlman came to my lab.’
Kuhlman, who today runs a protein design lab at the University of North Carolina (UNC) at Chapel Hill in the US, joined Baker’s team as a postdoc in 1999. Two years earlier researchers at the California Institute of Technology had reported the synthesis of the first protein fully designed by computer – a 28 amino acid-long sequence dubbed FSD-1. The protein replicated the shape of a naturally occurring zinc-binding protein, without requiring any metal ions to stabilise the structure.
Inspired by that work, Kuhlman was keen to focus on the design problem. ‘When I went to interview for David’s lab out in Seattle, I remember telling him about my interest in protein design … and he was really excited to get into the area as well,’ says Kuhlman. ‘And I remember right from the beginning, we were like “Wow, we should try and design something that’s never been seen before – that would be really cool.”’
Kuhlman got to work developing methods in Fortran and within a few months the team had a working program that, given a protein backbone, could predict a favourable amino acid sequence. ‘The next big challenge was then how do you create this novel backbone for a totally new fold,’ says Kuhlman. ‘And so that’s where we started melding the structure prediction capabilities of Rosetta with this sequence design program.’
The ‘Aha’ moment
The team settled on a target made of 93 amino acid residues that they named Top7. The Top7 protein was unique in that it had been intentionally designed to avoid folding patterns and amino acid sequences recorded in the Protein Data Bank – a repository of experimentally derived protein structures. In other words, Top7 was a completely new protein with no relation to any protein previously found in nature.
Another researcher in Baker’s lab, Guatam Dantas, who now leads a group at Washington University in St Louis, US, then led work to express the protein in bacteria and then isolate it. Top7 was then characterised by x-ray crystallography with help from Barry Stoddard at the Fred Hutch Cancer Center in Seattle, US.
‘When we solved the crystal structure, it was almost exactly like the design model,’ recalls Baker. ‘That was a super exciting moment – when we saw the crystal structure match up so well, it was really kind of amazing.’
By the time the crystallography results were in, Kuhlman had left Baker’s lab to start his own lab at UNC. ‘I got an email with the coordinates for the protein, and I could pull it up in a viewer and superimpose it with the design model. And there was this wonderful match between the two,’ he says. ‘I think that’s probably the single happiest day of my scientific career, in terms of an “Aha!” moment.’
Having proved that it was possible to design completely new proteins, Baker began considering the next question. ‘Top7 didn’t do anything – it was completely inert,’ he says. ‘So immediately then the challenge was: how do we design proteins to actually do things?’
In 2008, the group produced a collection of proteins that displayed enzymatic activity. These proteins were carefully drawn up to feature active sites that could promote a retro-aldol reaction on a substrate molecule not found in nature. While these enzymes displayed rather modest activity, the project marked the start of a new era of functional protein design.
Lucky turns
That same year, John Jumper had quit a PhD in physics and was considering his next move. ‘As an unemployed physicist, I did what people do and applied for finance jobs,’ he says. ‘In one of the lucky turns in my life, my application got forwarded from D E Shaw, the hedge fund, to D E Shaw Research, the molecular dynamics research group.
‘That’s when I first got into biology. And I pretty instantly fell in love with it, because I was working on really interesting, hard computational problems, but I could see all the way out to the implications for disease and health.’
After three years with Shaw, having taken his first steps into the protein folding field, Jumper returned to grad school. This time he moved to the University of Chicago, US, to work with biochemist Tobin Sosnick and theoretical chemist Karl Freed.
‘They had a joint lab and they worked on folding kinetics and the pathway problem of protein folding among many other problems,’ says Jumper. ‘I had originally joined, not so much because it was a protein folding lab, but because it was a mixed wet–dry lab. And I felt like, at that point, I needed to really understand what experimentalists do, how they do it, make sure I’m not in the computational bubble my entire life.’
While Jumper was working on his PhD, his future colleague Hassabis was making waves in the world of AI with DeepMind, a company he had co-founded in 2010. DeepMind blended two of Hassabis’ major interests – neuroscience and computer programming – working towards an ultimate goal of ‘solving intelligence’ and creating a true artificial general intelligence.
The company first started training its machine learning models on retro video games, showing that the AI could learn to play games like Pong and Space Invaders, eventually reaching an expert level. In 2016, its product AlphaGo made headlines by becoming the first computer program to defeat a top-level professional Go player.
Around this time DeepMind decided to turn its technology towards the protein folding problem. Jumper explains that Hassabis, who had a longstanding goal ‘to use AI to conquer scientific problems’ had been aware of the problem since his university days.
‘Totally by coincidence, every six months DeepMind would have a hackathon week where you work on whatever you want – and a couple of people like Rich Evans, who is still on the AlphaFold team, had searched “grand challenges in biology” and protein folding came up,’ Jumper explains. The DeepMind researchers spent a week trying to train a reinforcement learning agent to control Foldit – a video game created in Baker’s lab in 2008 and powered by Rosetta, that enlisted citizen scientists to help solve the protein folding problem. ‘That joined up to Demis’ long-term interest and became the AlphaFold project,’ adds Jumper.
Towards the end of his PhD, Jumper heard rumours about the fledgling AlphaFold project and applied for a job at DeepMind.
Gasps at Casps
Six months after Jumper joined the company, the DeepMind team entered Casp13. The competition uses a scoring method known as a global distance test (GDT), which provides a percentage accuracy based on how well a predicted structure matches up to a protein’s actual structure. While models had improved throughout the course of Casp’s history, for many years the GDT of winning programs had hovered around 30–40%.
In 2018, AlphaFold1 announced itself to the competition with a score of almost 60%. The system essentially used image recognition software to map protein structures stored in the Protein Data Bank and then used this information to train an algorithm to predict the unknown structures of new proteins.
Moult points out that at the time, several other groups had taken similar approaches, with some reaching accuracies comparable to those of the DeepMind team. ‘If you look at who had the best model for each target in 2018 it was more often DeepMind than anybody else, but other people were up there as well – so, it was a surprise, but not one that blew your socks off,’ he says. ‘What happened in 2020 was amazing.’
Shortly after Casp13, Jumper was appointed to head up the AlphaFold project. He explains that AlphaFold1 was ‘pretty close to a pure deep learning’ approach to protein folding. ‘We’ll take off the shelf computer vision networks, we’ll tune them well [and] do sophisticated optimisation,’ he recalls. ‘And I remember thinking, we’ve got to do more to bring in an understanding of protein physics.’
The team began testing a lot of different ideas – ‘some that were really incredible and some that were really boring’, according to Jumper – to gradually improve the accuracy of the AlphaFold model.
Importantly, Jumper notes, there was an ‘overall philosophical change’ in the way the system was developed. ‘For AlphaFold1, the neural network was the exact same as an image recognition network. Instead we tried to build what we understood about proteins, about evolution, biology, geometry, physics, into the structure of the neural network,’ he explains. ‘And so we started really designing new pieces of neural network technology exactly custom to this problem … And that turned out to be dramatic.’
A later analysis by the systems biologist Mohammed AlQuraishi at Columbia University, US, would show that AlphaFold2 requires just 1% of the training data from the Protein Data Bank to generate more accurate predictions that the original AlphaFold1 system.
‘So it’s a clean claim that building the machine learning around the protein problem was worth about 100 times extra data. And so it’s really about the insights you build into the deep learning,’ says Jumper. ‘I think the overall story is that people had built the deep learning of images and the deep learning of text, but no one had gone down and built the deep learning of proteins. And I think that’s probably the dramatic difference.’
When AlphaFold2 was entered into the Casp14 competition in 2020, it stunned the world of protein structure prediction. The system achieved average GDT scores of around 90% for difficult targets, putting the system on a par with methods used to determine protein structures experimentally.
Moult describes the astonishment that the competition’s organisers felt as they analysed AlphaFold2’s predictions: ‘We’re collecting the results one by one, and you see one that’s a hard target [and think] “Boy, that’s amazing!” and then you see the next one: “That’s extraordinary!” and then you see the next one: “What on earth has happened here? How can people suddenly be doing this with experimental accuracy?”’ he recalls. ‘So that was the real shock, the real excitement, the real breakthrough from our perspective.’
The AlphaFold2 results led the Casp14 organisers to declare the folding problem ‘largely solved for single proteins’. Other researchers were similarly taken aback. AlQuraishi wrote in a blogpost that ‘never in my life had I expected to see a scientific advance so rapid’, adding that ‘the core field has been blown to pieces’.
A bioscience revolution
Since the AlphaFold2 results were announced, both Baker’s lab and the DeepMind team have been at the forefront of using the technology to tackle major challenges in bioscience.
Almost immediately the DeepMind team began using AlphaFold2 to solve the structures of as many proteins as they could find. In 2021, they published a database containing 350,000 predicted structures, including structures for every protein found in the human body. A year later the team released a further 200 million structures covering almost every protein known to science.
Within a few months there were already discoveries being made with AlphaFold
The resource has had an immediate impact for the global academic community, with over two million researchers having used the database to date. In just three years, the DeepMind team’s main paper on AlphaFold2 has been cited 16,000 times according to Web of Science. ‘What really shocked me is the degree to which it was already entering people’s science – within a few months there were already discoveries being made with it,’ says Jumper.
‘I remember this special issue of Science on the structure of the nuclear pore complex … with something like 100 mentions of AlphaFold, and we were uninvolved – and it was showing up a year after our paper came out,’ he adds. ‘I think really that was the moment at which I knew how much science people were doing without us.’
Baker’s team was itself quick to learn from the advances made at DeepMind. ‘When the AlphaFold results were released there was a lot of gloom and doom among some more established professors, saying “OK, now tech companies are going to totally dominate structural biology, and what were academics left to do?”’ says Baker. ‘But I was really excited. We just figured we had to learn how to do this kind of thing.’
DeepMind eventually released the source code for AlphaFold2, but for several months it was unclear whether this would be the case. In the meantime Baker’s postdoc Minkyung Baek, who now runs her own lab at Seoul National University in South Korea, set about trying to unpick the program and apply some of its underlying techniques to Rosetta.
‘David felt like we had to develop our own version if we really want to extend that protein structure prediction method for another application such as protein design – we needed a tool we really understood and that we could easily change or modify for our own purpose,’ she says.
By incorporating some of the ideas behind how data was fed to AlphaFold2 and how the program processed spatial information, Baek quickly created Rosetta Fold achieving prediction accuracies close to those demonstrated by DeepMind’s product. Importantly, Baker’s team was able to use what it had learned to take its protein design work to the next level.
The team set about building new AI-powered protein design tools that have dramatically improved the sophistication and accuracy of the proteins the lab sets out to create. One such program is a generative AI model called RFdiffusion. ‘It basically finetunes RosettaFold for protein structure generation,’ explains Baek. ‘And it works beautifully – that was a big advance for a protein design.’
RFdiffusion and another package called ProteinMPNN have proven to be particularly powerful for designing proteins that bind to other proteins and molecular targets.
‘What the generative AI methods let you do that we couldn’t do with Rosetta is to provide a kind of minimalist description of the function you want, and then the generative AI builds up the protein structure that will have those properties – if you carry out diffusion in the presence of a target, you get a protein that has a perfectly complementary shape to the target,’ says Baker. ‘If you describe a catalytic site, then the diffusion process will build a protein around that site. And so, where before we had quite low activities, now we’re getting quite high activities straight out of the computer, which is really exciting.’
These designer proteins are no longer just scientific curiosities, but are showing promise for tackling real world problems. In 2022, South Korea approved a Covid-19 vaccine based on self-assembling protein nanoparticles designed at Washington by Neil King, one of Baker’s former postdocs. Meanwhile Baker’s lab recently posted a preprint on designed proteins that bind to snake venom and that appear to work as antidote to snakebites.
DeepMind too, is continuing to bring out new products. In May this year, the company reported on its AlphaFold3 model that is capable of predicting structures of complexes of proteins with many other biomolecules, including nucleic acids, sugars and antibodies. The company announced the report by claiming that AlphaFold3 ‘predicts the structure and interactions of all of life’s molecules’.
Jumper acknowledges that using the software for these tasks is ‘more hit and miss’ due mainly to the fact that less training data exists for molecules like RNA and DNA than for proteins. However, he is confident that the company will continue to push the boundaries of what can be achieved with its software. ‘What we really have to do is refine and, as I like to jokingly say, deep learn harder – come up with often clever ideas, but really just improve overall accuracy,’ he says. ‘And then all these subtle effects, I think, will ultimately snap into place.’
What else can AI answer?
Although the original protein-folding question appears to have been answered, Moult points out that there are still new challenges to tackle. ‘What other problems might fall to these methods?’ he asks. ‘The really obvious one is RNA structure … another area where there has been a lot of hype is calculating how drug-related molecules bind to a protein.’ Another topic that remains to be fully addressed is proteins that take on multiple structures.
Many observers have also queried the somewhat mysterious way in which AI solves protein structures, noting that programs like AlphaFold are essentially a black box – they provide answers, but we don’t really know how they do it.
It’s a question that intrigues Hassabis, who at a press conference following the Nobel announcement noted that ‘we’re at a fairly early stage of building the right analysis tools for these things’. He wonders whether it’s possible to borrow techniques from neuroscience and apply them to this new problem. ‘What’s the equivalent of an fMRI scan for an AI system – a virtual brain, if you like?’ he asked. Hassabis also raised the possibility that perhaps in the future, systems like AlphaFold could be built with the capability to explain themselves using natural language in the way a human scientist would.
At the same event, Jumper said he was humbled that each data point that AlphaFold was trained on probably represented years of work carried out by a PhD student somewhere in the world. But while generations of researchers’ work made AlphaFold possible, Jumper marvels that AlphaFold itself can now return the favour. ‘The moment that I will be almost as excited as this, will be the Nobel prize that talks about work done with AlphaFold to understand a disease – I can’t wait for that moment.’
Jamie Durrani is senior science correspondent at Chemistry World