A Blog by

There’s No Plague on the NYC Subway. No Platypuses Either.

There is no good evidence that Yersinia pestis—the bacterium that causes plague—is riding aboard the New York City subway. That’s the message from several microbiologists, in response to a wave of news stories that emerged last Friday.

“Plague, anthrax and cheese? Scientists map bacteria on New York subway,” said the Guardian. “From beetles to bubonic plague: Bizarre DNA found in NYC subway stations,” proclaimed the Washington Post. “Terrifying microbe map of New York’s subway system reveals superbugs, anthrax and bubonic plague,” blathered the Daily Mail, duly retaining its crown as the champions of scaremongering.

All of these stories were based on a census of some of New York City’s smallest residents—its microbes. Over the course of 18 months, a team of scientists led by Christopher Mason from Weill Cornell Medical College swabbed surfaces all over the city, including every open subway station (see a video about their subway sampling here). They then analysed the DNA in their samples to identify the microbes that lived on each surface.

There were plenty of interesting results. For example, the bacteria from South Ferry Station, which was flooded by Hurricane Sandy in 2012, were closer to marine microbes than to those in the rest of the subway. But the team’s most notable claims were that they found DNA from the plague bacterium Yersinia pestis in three samples, and from the anthrax bacterium Bacillus anthracis in two. These nuggets predictably wound their way into every major news story.

The researchers downplayed the significance of these results, saying that if Y.pestis was present, it was unlikely to be “active and causing disease in people”. As Mason told the New York Times, “We’re saying there’s evidence for these things… but no one should worry.”

But several microbiologists think that even this statement goes too far. “[They] have not provided persuasive evidence that the agents that cause plague or anthrax are present anywhere in the New York City subway system,” says Ian Lipkin, a well-respected virus hunter from Columbia University. “The genetic footprints they report are not specific for the agents that cause anthrax or plague; they are also found in other common bacteria that are not associated with disease.”

The team arrived at their conclusions after compiling the DNA in each sample, breaking it down into smaller pieces, and then sequencing the fragments. They then searched for these sequences—or “reads”—in a public database of genes from all known organisms. If they found many matches for a given microbe, they concluded that said microbe was present in their samples.

“This method is notoriously unreliable,” says Willem van Schaik from Utrecht University, who studies antibiotic-resistant bacteria. As Lipkin also noted, it’s prone to false alarms, because a given read could match DNA that’s found in many other bacteria besides Y.pestis.

Rob Knight, formerly at the University of Colorado Boulder, showed just how ludicrous this problem can get last year. His colleague, Andrea Ottesen at the FDA, swabbed tomato plants in a field in Virginia, and Knight analysed the DNA in those samples. He found matches to the duck-billed platypus—an Australian animal, not known to live in Virginia. They then analysed over 19,000 publicly available microbiome samples from around the world; around a third threw up matches for platypus DNA. Either the platypus secretly rules the world or, more likely, this was a hilarious case of false positives gone mad. The team, including Antonio González Peña, even created a programme called Platypus Conquistador to rectify the problem.

There are signs of similar problems in the subway paper. In the case of Y.pestis, the team found several reads that matched a plasmid—a free-floating ring of DNA that sits outside the bacterium’s main genome. They highlighted one particular section of the plasmid in one of their figures. Based on this, Van Schaik did his own search and found that sequences in this section are also found in at least three other bacterial species.

Mason’s team also identified DNA from many eukaryotes—that’s animals, plants, and other complex organisms—and listed the top species in one of their tables. They were, starting from the top: the mountain pine beetle, which lives in the west coast of North America; the Mediterranean fruit fly, which does not exist in the continental US; the cucumber; and humans. Hmm. The fly, in particular, is a major agricultural pest and the subject of intensive surveillance. Its presence in New York is “so unlikely that I think the approach they used is just flawed,” says Van Schaik.

Unlike the fly, Y.pestis does exist in the US, but in the southwest where it infects rodents. It is a stranger to New York. Lipkin’s team have done extensive surveys of the city’s rats and failed to find Y.pestis in any of them.

To convince their critics, Mason’s team would have to show that reads from the subway analysis map to the entirety of the Y.pestis plasmid. Then again, plasmids can easily move from one bacterium to another, so it would be even more convincing to show reads that map to Y.pestis’s actual chromosome. “Even better would be to prove the existence of Y. pestis through some independent means, such as culture,” says Nick Loman from the University of Birmingham, who studies the genomes of disease-causing microbes. By “culture”, he means trying to actually grow bacteria from collected samples.

I contacted Chris Mason about these criticisms and he is preparing a blog post to address them, and others that he has received. [Update 18 Feb: Mason has published the post and it is humble, informative, detailed, and introspective]

As I mentioned, the paper has other interesting results and it might seem churlish to pick on this one. But it symbolises some of the problems in the study of microbiomes—the collections of bacteria that live in specific animals or environments. Microbiome research is among the hottest fields in biology and is attracting hordes of enthusiastic scientists. This is great—(disclosure: I’m writing a book on animal microbiomes)—but the field also risks throwing up a lot of misleading and false conclusions if methods aren’t applied properly and results aren’t analysed cautiously. (Last November, I wrote about another common problem that might lead to false positives in a lot of microbiome studies.)

One could argue that these issues come out in the wash, and that science corrects itself. Indeed, Mason told the New York Times that not reporting the fragments of anthrax and plague “would have been irresponsible”. Then again, readers were hit with a wave of headlines that raised the possibility of plague on the subway. “If I wasn’t a microbiologist, I would be scared by this and rightfully so,” says Van Schaik.

Marc Lipsitch, an epidemiologist from the Harvard School of Public Health, agrees, and adds that panic-quelling follow-up stories, like this one, don’t help matters. “Stories that present a highly hedged finding as news, but then say “Don’t worry”, make scientists seem like we aren’t telling the whole truth,” he says. “Either it is news, or we shouldn’t worry, but it’s tough to see how both could be true.”

Update: Nick Loman has put up a short post illustrating one of the problems I talked about in this post. He took E.coli, shred its DNA, sequenced the pieces, and then matched them up to databases. By right, 100% of the reads should come out as E.coli. In fact, just 61% of them did.

Reference: Afshinnekoo et al., Geospatial Resolution of Human and Bacterial Diversity with City-Scale Metagenomics. 2015. Cell Systems http://dx.doi.org/10.1016/j.cels.2015.01

Correction: The article was amended to note that Andrea Ottesen swabbed the tomato plants, rather than Rob Knight himself.

7 thoughts on “There’s No Plague on the NYC Subway. No Platypuses Either.

  1. Everything can be proven and disproven I guess. Its all up to a person’s eloquence and ability to convey ”their truth” in a way that will make the listener believe it. Really few of us have a background in biology solid enough in order to say if Doc Mason is just saying a fallacy. The trains in nyc are dirty, really really dirty. Maybe its not the black plague but there’s definitely all sorts of crap.

  2. It’s an impressive study, but it’s unfortunate that Afshinnekoo et al. didn’t sequence any of their 51 negative control samples (i.e., collection blanks). Page 13 of their paper describes these blanks and how they tested just 13 of them using only coarse quantification of total DNA. In the context of papers showing how contamination is so pervasive and under-observed (refs below) it would have been great if they had analyzed the blanks as carefully as they analyzed the subways. This adds expense and difficulty, but Nguyen et al. (2014) describe how to do it. I think robust negative controls are fast becoming essential for deep sequencing.

    Xu et al. 2009 doi:10.1002/elps.200800473
    Champlot et al. 2010 doi:10.1371/journal.pone.0013042
    Nguyen et al. 2014 doi:10.1111/nph.12923
    Lusk et al. 2014 doi:10.1371/journal.pone.0110808
    Salter et al. 2014 doi:10.1186/s12915-014-0087-z)

  3. Genomic studies looking for environmental-DNA can be more realistally assessed if they apply at least one other corroborating diagnostic test of ‘presence’ – for example selective microbiological culture. The potential for false positive detections (Type 1 error) based on a single test with unassessed specifity applies to these new technologies as it does with other diagnostic tests.

  4. Yet another example that shows that we are looking at microbes with the wrong magnification lenses, that is, their DNA. The presence of DNA does not mean that your microbe is alive. Not even that your sequence is an organism at all! Besides, dead microbes are not pathogens, and there is a lot of dead stuff out there, especially in dry environments. Show it is alive/viable, and then come with conclusions.

  5. People tend to pick and choose the science they will believe, typically based on politics and convenience but also based on whatever the media tell them matters. That’s a risk whenever the popular media are the conveyance for information about scientific news, and you’d think that scientists would be far less vulnerable to that kind of persuasion. You’d also be wrong. How many people, and how many scientists, laugh at the discredited work purporting to show a link between autism and childhood vaccination, yet down megadoses of Vitamin C at the first sign of a cold?

Leave a Reply

Your email address will not be published. Required fields are marked *