A Blog by Ed Yong

Computer gamers solve problem in AIDS research that puzzled scientists for years

When scientists struggle with a problem for over a decade, few of them think, “I know! I’ll ask computer gamers to help.” That, however, is exactly what Firas Khatib from the University of Washington did. The result: he and his legion of gaming co-authors have cracked a longstanding problem in AIDS research that scientists have puzzled over for years. It took them three weeks.

Khatib’s recruits played Foldit, a programme that reframes fiendish scientific challenges as a competitive multiplayer computer game. It taps into the collective problem-solving skills of tens of thousands of people, most of whom have little or no background in science. Here’s what I wrote about Foldit last year:

The goal of the game is to work out the three-dimensional structures of different proteins. Proteins are feats of biological origami; they consist of long chains of amino acids that fold into very specific and complicated shapes. These shapes can reveal how proteins work, but solving them is fiendishly challenging. To do it, scientists typically need to grow crystals of purified protein before bouncing X-rays off them.

Foldit takes a different approach, using the collective efforts of causal gamers to do the hard work. And its best players can outperform software designed to do the same job. Best of all, you don’t need a PhD to play Foldit. Barely an eighth of the players work in science, and two-thirds of the top scorers have no biochemistry experience beyond high school. The controls are intuitive; tutorial levels introduce the game’s mechanics; colourful visuals provide hints; and the interface is explained in simple language. While protein scientists concern themselves with “rotating alpha-helices” and “fixing degrees of freedom”, Foldit players simply ‘tweak’, ‘freeze’, ‘wiggle’ and ‘shake’ their on-screen shapes.

Foldit’s success relies on the fact that it doesn’t shallowly flirt with interactivity – it’s a true game. Its creator Seth Cooper designed it to “attract the widest possible audience… and encourage prolonged engagement”. It’s competitive: players are scored based on the stability of the structures they end up with and a leader board shows how they rank against other gamers. There’s also a social side: gamers can chat on online forums, work in groups to solve puzzles and share solutions on a wiki. And just like real game development, everything was tuned according to feedback from the players. Tools were added and refined, the difficulty of the tutorials was tweaked to stop frustrated beginners from leaving, and puzzles were matched to the skills of the players.

There’s the thrill of contributing to genuine scientific research, but that motivates less than half of the community. The rest do it for the achievement, the social aspects and largely, because the game was fun and immersive.

Foldit’s origins lie within Rosetta, a piece of software designed to solve protein structures by simulating and testing thousands of different folds. Rosetta is an example of ‘ distributed computing’, where volunteers run the program on their home computers when they don’t need it. They effectively donate their computing power to speed up the laborious task of solving protein structures. But the volunteers wanted to use their biological computers – their brains – as well as their man-made ones. They suggested an interactive version of the programme and in May 2008, they got their wish with Foldit.

Last year, Cooper showed that Foldit’s gamers were better than the Rosetta programme at solving many protein structures. They used a wide range of strategies, they could pick the best places to begin, and they were better at long-term planning. Human intuition trumped mechanical number-crunching.

This year, Khatib wanted to see if the Foldit community could solve fresh problems. He entered the players into a twice-yearly contest called CASP (Critical Assessment of Techniques for Protein Structure Prediction), where structural biologists from all over the world compete to predict the structures of proteins that have almost been solved. They get the best predictions from Rosetta to begin with. Then, they’re on their own.

Khatib’s gamers, bearing names such as Foldit Contenders Group and Foldit Void Crushers Group, had varying degrees of success in the contest. In many of the categories, they did reasonably well but they couldn’t match the best groups. They weren’t as good at using the structures of similar proteins to tweak the ones they were working on. They could also head down dead ends if they started at the wrong place. In one case, their strategy of refining their starting structures to the best possible degree led to one of the “most spectacular successes” in the contest. But mostly, they focused too heavily on tweaking already imperfect solutions that other teams achieved better results by making large-scale changes.

Learning from that lesson, Khatib stepped in himself. He agitated the initial protein structures in many random ways, to create a wide variety of terrible answers that the gamers could then refine. In their attempts, they came up with the best-ranked answer to the most difficult challenge in the competition.

It was a success, and more would follow. After the competition, the players solved an even more important problem. They discovered the structure of a protein belonging to the Mason-Pfizer monkey virus (M-PMV), a close relative of HIV that causes AIDS in monkeys.

These viruses create many of their proteins in one big block. They need to be cut apart, and the viruses use a scissor enzyme –a protease – to do that. Many scientists are trying to find drugs that disable the proteases. If they don’t work, the virus is hobbled – it’s like a mechanic that cannot remove any of her tools from their box.

To disable M-PMV’s protease, we need to know exactly what it looks like. Like real scissors, the proteases come in two halves that need to lock together in order to work. If we knew where the halves joined together, we could create drugs that prevent them from uniting. But until now, scientists have only been able to discern the structure of the two halves together. They have spent more than ten years trying to solve structure of a single isolated half, without any success.

The Foldit players had no such problems. They came up with several answers, one of which was almost close to perfect. In a few days, Khatib had refined their solution to deduce the protein’s final structure, and he has already spotted features that could make attractive targets for new drugs.

“This is the first instance that we are aware of in which online gamers solved a longstanding scientific problem,” writes Khatib. “These results indi­cate the potential for integrating video games into the real-world scientific process: the ingenuity of game players is a formidable force that, if properly directed, can be used to solve a wide range of scientific problems.”

Update: Stephen Curry, who works on protein structures, had this to say about the paper: “Credit where it’s due: this is certainly an innovative approach to the problem of determining crystal structures of proteins. And I do like the idea of ‘citizen science’. Although it’s probably questionable how much science the gamers are understanding, the involvement in this sort of research, even if it is just at the level of playing a game, is undoubtedly a good thing.”

Curry also points out that a structure for this protein was published in 2003 using a different method called nuclear magnetic resonance. Khatib says that this is “quite inaccurate” and that people have struggled to use it to progress any further, but Curry says that they don’t say much about the differences between the old and new structures.

Likewise, Khatib doesn’t mention how closely related the M-PMV protease and the HIV ones are. “This information is crucial for deciding whether a structure of M-PMV protease is going to be any use as a template for the design of novel classes of drug targeted to HIV protease. If I had reviewed this paper, I would have asked for that information to be included because it is needed to make sense of observed differences in structure,” he says.

Reference: Khatib,  DiMaio, Foldit Contenders Group, Foldit Void Crushers Group, Cooper, Kazmierczyk, Gilski, Krzywda, Zabranska,  Pichova, Thompson, Popović, Jaskolski & Baker. 2011. Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nature Structural and Molecular Biology http://dx.doi.org/10.1038/nsmb.2119

More on Foldit: Foldit – tapping the wisdom of computer gamers to solve tough scientific puzzles

29 thoughts on “Computer gamers solve problem in AIDS research that puzzled scientists for years

  1. Very nice work, but considering that so many citizen scientists contributed to the solution of the puzzle, I would have expected the journal to make the paper freely available so that many more citizen scientists would have been able to read it. A pity.

  2. This is cool news. The human brain is an amazing machine. It is truly in-credible that both the human brain and the simple molecules of these viruses can be so complex and developed by so many perfect accidents all the time over billions and trillions of years.

  3. I realize it’s voluntary, but it sounds awfully much like Matrix to me. Are any of these gamers going to be compensatedweeded–and if the big pharm. companies start using the results?

  4. @Kim,

    Very unlikely.

    None of the gamers even received credit in the Nature paper despite their input being central to the eventual refinement of the structure.

  5. @Mary – This is personal blog that’s hosted at Discover. I write in my spare time and I often don’t get a chance to proofread thoroughly. Your suggestion to hire an editor is sweet but unlikely.

    @Kim – On what grounds would they be compensated? This is, as you say, an entirely voluntary exercise.

  6. The gamers ought to be given credit in the nature article, even if only mentioned by their gaming handles. To do otherwise is unscholarly and a borderline crime in academia.

  7. If only problems like the economy, etc. could be solved this way. We need to find ways like this to keep utilizing the collective brain power of vast numbers of people… That’s the key to the future.

  8. @Groos

    Many of the problems with the economy have multiple solutions–it’s not a matter of figuring out how to fix them. Rather, the issue lies in getting different parties to agree to a common solution, and then putting that plan into effect.

    I agree, though, that collective brainpower is the most effective method of solving large-scale problems.

  9. The gamers are given credit in the article. They’re listed as Foldit Contenders Group, Foldit Void Crushers Group. You perhaps want them to be listed by name individually. This seems to me like it would be pretty unusual practice for any academic publisher. Science involves the work of many people such as lab technicians who don’t get credit – authorship is reserved for people who make intellectual contributions to the work. Is that the case here (or indeed, for other citizen science projects)? I think not. Note that most of the gamers don’t understand the underlying science at all.

    I grant you that it’s an interesting issue to consider in terms of acknowledgments, but the players are acknowledged twice – once in the credits and once as authors. That’s actually quite a lot.

  10. I have to agree with Ed. The teams were listed by name as well as the varying degrees of success they performed at. If individual acknowledgements are what people feel are missing, it’s possible that the entire list of participants would span pages much more numerous than the article itself. Unless it’s a special printed edition for that specific event, I don’t see any publisher doing that. As for the outstanding individuals who volunteered to take part, I assume it would have been made clear what they would and would not receive for their participation. To volunteer for something and then to be somehow shortchanged or overlooked while your efforts are being utilized doesn’t make sense to me. I’ve worked with lots of volunteers and they generally don’t take that stance unless someone else is able to put the idea in their head.

  11. Now if Firas Khatib from the University of Washington would suggest that the business and Polly-sci departments put together a game, Washington Tycoon, we could turn our country around based on citizen-wide, gamer-play.
    Just think, no more congress or any need for the fed or any other gov. organizational structure. Just feed the gamers their parameters and let them/us rock!
    Run an entire country, AN ECONOMY Man!….the nuclear football, taxation, war, federal and state issues, the postal service, roads, bridges, transportation, foreign policy, CIA, FBI………WOW!!! Turn it over to the people through a game!
    Awesome idea!!!

  12. I wonder if the same approach can be applied to high-temperature superconductor -problems? Same kind of optimization algorthms exists for these problems. Adding some human intelligence in the form of Fold It , might present new avenues for researchers to follow.

  13. Yeah … Khatib doesn’t really mention that the structure for HIV protease is already known and that structures of protease inhibitors docking with HIV protease has already been published. I love the fact that he exploits HIV hype to promote his own “research”. It should be stressed that gamers, although brilliantly, only completed the last mile. Solving protein structures is a little harder than shooting down rockets, dear author.

    (PS: Dear, Khatib: How about solving the structures of PrP^c and PrP^sc? )

  14. Too many people are simply writing this off to either “online gaming” or “crowd sourcing.” What you also might want to consider, is that this particular group of Foldit players are “naturally talented puzzle/problem solvers”, that are fascinated by science. And in this case, this particular group of puzzle solvers seem to be fascinated by structural & molecular biology – not just any science, but specifically, structural/molecular biology!BRAVO to Foldit and this group of puzzle solvers!! What a GREAT story!! Sadly however, it illustrates how many of us are in the wrong role!! “The experts” have been working on this problem for a decade, and the “puzzle solvers” solved the puzzle in 3 WEEKS!!!While I’m confident that “the experts” are very bright and talented, they are clearly NOT “naturally talented puzzle solvers.” Either that,, or they’re not naturally fascinated by structural/molecular biology… Without knowing them, of course I have no idea if “the experts” should even be in this field, but one could argue that some of them certainly don’t seem to be in the right role.. And what about these particular puzzle solvers… What are they doing for work? What fields are they in? What are they naturally most fascinated by? And, are they leveraging their natural talent every day to help solve similar problems? My argument is; when you marry one’s Top Natural Talent with their Greatest Natural Fascination, it can lead to great discoveries – In every area of endeavor! Using Einstein as an example, he used his considerable math talent to answer his own most profound questions about space and time. All “The Greats” focused their Top Natural Talent on their Greatest Natural Fascination, to help change the world, and you can to! So let’s raise the bar for ourselves and one another; Get in the right role in the right industry to do YOUR BEST WORK! Find your Top Natural Talent and put it to work on your Greatest Natural Fascination!

  15. @greg kellerman, “Now if Firas Khatib from the University of Washington would suggest that the business and Polly-sci departments put together a game, Washington Tycoon, we could turn our country around based on citizen-wide, gamer-play.”

    We already have that framework in place. It’s called “Capitalism”.

    Unfortunately, the players in another group called “Government” has imposed so many restrictions, taxes and other disincentives on the “Capitalism” group that its citizen-wide creative activities have been stifled, with the economy suffering as a direct result.

    It doesn’t help that the “Government” group thinks that their function is not to promote the economy, but to support a third group, the “Drones”, whose members believe that they deserve life-long support simply for voting some of their more vocal members into the “Government” group, rather than participating in the “Capitalism” group.

  16. Hardly the place to attempt to disguise a right wing diatribe as gaming comment. A more legitimate suggestion would be a game that incentivizes all three corners of our democratic system — capitalism, government, and social justice, to obtain the best possible outcome for all three layers of society: the poor and homeless, the working class, and the wealthy. Now that would be a game worth playing.

  17. @Kim, Im glad you brought up the issue of compensation and credit. Even if it did end up getting dismissed.

    It is something that needs to be addressed at some point. We live in a society that believes that “value creators” have the right to profit from that, and this type of “game” does seem to be an exploit of that. Perhaps it will need to be addressed with the gamers, not the people exploiting their gaming, but it should be addressed at some point in some way.

    Our economic system is already one that favors heavily those who are adept at exploiting the talent, work, and innovation of others, rather than those who are talented, hard working or innovative, and as a species we do need to ask ourselves if its wise to set up systems that favor exploitation over talent.

    We are playing the game of natural selection, after all, and clearly, the exploitative are “winning” on a global scale. Im not sure thats in humanitys best interests in the long run.

  18. I think if gamers want credit for contributing to science then gamers should go to school, get a degree, and do the hard work of becoming a scientist just as every other scientist does. Nobody is stopping them from doing this.

  19. well done! reminds me of an episode on south park. as for the naysayers, complainers and the like: perhaps they should use their time solving problems rather than critiquing those who already have.

  20. Whether or not the structure had already been solved using NMR, this is an interesting method for finding solutions to a fairly complex problem. I appreciate the novelty of its approach, and in fact I feel the full implications have not even been brought to light. Consider this; in order to convince people to work on this problem, a dedicated game had to be created that provided adequate gratification to each player (whether they had the slightest interest in the scientific implications or not), a series of tutorials to train the player how to make modifications to the proteins structure in a way that would yield stable final products, and then unleash the newly trained user on some already solved, more complicated problems to verify that they had properly learned to solve this type of problem. From there, they finally got the opportunity to work on new problems which had not been fully solved previously. This procedure is remarkably similar to another type of problem solving strategy— one that is neither a brute force computational method, nor a sheep herding exercise for gamers. What I’m talking about is using neural networks (which require some sort of fitness function for feedback on their solutions, training data to use so they can adjust to create the proper outputs, and then can be finally used to solve complicated non-linear problems such as these) instead of a collection of gamers with some free time on their hands. Neural networks can be trained specifically for one task, do not need to be convinced of the importance of a problem (or gimmicked into being involved with video game tropes). Furthermore, they can work 24 hours a day, 7 days a week and only at the cost of computing power. So, I suppose the main question is what size and depth of a neural network is needed to solve such complicated protein folding problems and therefore how much computational resources would each network require. It’s possible though that such problems could more efficiently be solved by a mass of such networks than by the methods described in this paper.

    Also, in response to some people saying that the gamers should get more credit or that the gamers seem to be more skilled at solving problems than the scientists in charge, one thing to consider is the fact that someone had to figure out the rules for the game and then create it before the gamers could even touch it. If all it took to be a researcher was to be skilled at solving brightly colored puzzles on a computer screen then it wouldn’t require 4 years of bachelor work and 6 years of PhD work (plus post-doc and all that good stuff). That’s not to say they aren’t skilled, it’s just that there are many many other cogs that make this machine work than the final step of brute forcing solutions.

  21. Thus, this could be a god sign that everything has a solution and this game could be a basis in solving scientific problems. Thanks for bringing this wonderful ideas.

  22. Here’s a comment from Firas Khatib:

    As one of the authors on the NSMB paper (which you can read for free right now as it was NSMB’s Article of the Month: http://www.nature.com/doifinder/10.1038/nsmb.2119) I wanted to address Stephen Curry’s great point that a lot more information about the actual M-PMV protein would have made for a more complete paper.

    All those details can be found in this paper (where we paid the open access fees so that anyone can read it for free): http://journals.iucr.org/d/issues/2011/11/00/lv5014/index.html

    You can see that the structure of HIV-1 protease DIMER has been known for 23 years now. But there was no accurate structure of retroviral protease in the MONOMERIC form: before the protein actually assembles as an enzymatically competent dimer. The crystal structure that the Foldit players helped solve fills this long-time gap. Hopefully now, with the knowledge of the structure of the M-PMV protease monomeric target, we can prevent dimerization and we’ll have an excellent new avenue to new-generation AIDS drugs. (And obviously we have deposited this result in the publicly-accessible Protein Data Bank, so that anyone can access it: http://www.pdb.org/pdb/explore/explore.do?structureId=3SQF)

    I also wanted to point out that (in terms of the players getting proper credit on the Nature Structural and Molecular Biology paper) we actually asked the 3 individual players to be co-authors on the paper but they refused, instead asking for their entire team to get the credit.

    You can see ‘mimi’ mentioning this in the cosmiclog piece (https://bitly.com/foldit1): “You may be aware that we asked for accreditation for the Foldit Contenders Team within the article, rather than being named individually.”

    We were amazed by this display of camaraderie!

Leave a Reply

Your email address will not be published. Required fields are marked *