A Blog by

Do You Really Understand Why Water Boils? New Survey Says, Probably Not.

Good news, America: Roughly 75 percent of us know who developed the polio vaccine, that ocean tides are caused by the tugging of the moon’s gravity, and that astrology and astronomy are, in fact, different things. Phew.

Great news! The vast majority of us (86 percent) know the Earth’s core is hotter than its surface, and that if you want to make nuclear weapons, you’re going to need some uranium (82 percent of you know that, which could be somewhat concerning).

The not-so-good news: Two-thirds of you don’t understand why water boils, and figuring out what makes sounds loud is hard.

These are just some of the results released today from a science knowledge survey conducted by the Pew Research Center. Over three weeks in 2014, 3,278 people answered 12 science questions. Overall, 6 percent of respondents got a perfect score, and half answered more than eight questions correctly. Not surprisingly, the number of correct answers lined up closely with education level.

I guess that’s not so bad, especially given that some of the questions were a bit off the beaten track, like asking what a light-year measures (Time? Distance? Brightness? Weight?) or how to interpret a graph with tooth decay on one axis and sugar consumption on the other (take a guess).

Or were they? Based on these results, it might be tempting to think we can relax and stop worrying about Americans performing somewhat dismally in the sciences; but before we do that, we need to take a good look at what the Pew survey — and others like it — are really testing.

The key with such surveys, says the University of Michigan’s Jon Miller, who’s been studying science literacy for nearly four decades, is to ask questions about core concepts. Things like what molecules are, what DNA is, and how the universe is organized. Show people an image of a spiral galaxy, like the Milky Way, and ask if they know what it is and why it’s important.

Conversely, asking about names, dates, or places can tap into someone’s biographical rather than foundational knowledge. “It’s not really getting at whether they have a skill level,” Miller says.

In other words, surveys should not be testing whether people understand the headlines in today’s science stories, but whether they have enough basic knowledge to understand the headlines 20 years from now.

“What you’re not trying to do is measure, do they know what the SARS virus is, or what the West Nile virus is — but do they understand what a virus is?” Miller says. “What we should be doing is thinking about measuring the constructs that people need to know.”

Some of the questions on the Pew survey might do that. Miller points to the question about what a light-year measures (“it offers a basic structural sense of the Universe”), and the scatterplot, which tests whether people can interpret the relationship shown on a graph.

But others might be a bit more like trivia, instead offering a snapshot of this particular moment in time rather than true science comprehension. It’s a pitfall that many surveys have run into, starting with the very first one, conducted by the National Association of Science Writers in 1957. One of the survey questions asked about radioactive fallout, and which type of element commonly rained down from the sky. Would you know the answer now? Maybe not. But in the mid-20th century, Miller notes, the Nuclear Test Ban Treaty hadn’t been signed and nations were still exploding bombs overhead. So, newspapers regularly covered strontium-90 fallout.

Another example, perhaps, is a question on this year’s Pew survey. It shows a photo of a comet and asks people whether the icy object with “a tail of gas and dust that extends millions of miles” is a comet, star, asteroid, or moon. Seventy-eight percent of people answered correctly. Coincidentally, the European Space Agency’s Rosetta mission — in which a spacecraft pulled into orbit around a comet and then sent a lander to the dirtball’s surface – was heavily featured in the popular press around the time of testing (Aug. 11-Sept. 3, 2014).

One more result that might speak to this point comes from a Pew question about Jonas Salk, who developed the polio vaccine in the early 1950s. Among older adults who were alive at the time (65 years and up), 86 percent got the question right. Conversely, only 68 percent of young adults, between the ages of 18-29, got the question right.

In these cases, is it possible to disentangle actual science comprehension from biographical memories or simply reading the news? It’s not clear. And, might it be more important to understand how vaccines work and how Earth’s solar neighborhood is put together? Pew notes in its report that “how much Americans appear to know about science depends on the kinds of questions asked,” and that the survey only represents “a small slice of science knowledge.”

It would be interesting to see how the answers to these questions fluctuate over the next five years, or 10 years, and if popular science knowledge waxes and wanes like the phases of the moon…you do know how that works, right?

10 thoughts on “Do You Really Understand Why Water Boils? New Survey Says, Probably Not.

  1. In the online version of the survey I answered the polio vaccine question correctly. Not because I remembered Jonas Salk’s name but because I knew the other 3 were physicists (and one chemist who specialized in the part of chemistry that’s almost physics).

  2. I am given to understand the moon has two sides, dark and light. The moon doesn’t spin so as it orbits the Earth we sometimes see the dark side and sometimes the light side. The light side gives us a full moon and the dark side a new moon. Of course sometimes the spring tides cause earthglow off the surface giving us a blue moon, but that is rare.

  3. In addition to mixing random factoids with serious scientific knowledge, the survey itself contained a major scientific error. Question 10 asked “Which of these elements is needed to make nuclear energy and nuclear weapons?” The possible answers were N, NaCl, CO2, and U. Two of the possible answers are not elements.

    Sigh.

  4. > how the universe is organized. Show people an image of a spiral galaxy, like the Milky Way, and ask if they know what it is and why it’s important.

    Is it REALLY important to know? I think it’s important that we don’t really know, we learn more by asking questions and wondering, than in the knowing. The spinoffs are more important than the knowledge we are in a quest for.

    The fine detail of the big bang, what ACTUALLY happened, and whether the cosmos dies in a big crunch, or thins out to a fine mist at 1/293928158139 K, they are unimportant in our lifetime, and I suspect in mankind’s lifetime. And that is not being negative, that is the most positive take I can think of

  5. Ha-ha! Kirk! I hope kudzu is kidding too. That has to be a joke! & this survey is faulty. I feel like I could have put together a better survey to get closer to the truth.

  6. Regardless of the results, the actual question on the test is
    “Denver, CO is a higher altitude than is Los Angeles, CA.

    Which of these statements is correct?
    1. Water boils at a lower temperature in Denver than Los Angeles
    2. Water boils at a higher temperature in Denver than Los Angeles
    3. Water boils at the same temperature in both Denver and Los Angeles

    This question is considerably different than the phrasing you’ve used in this article. “Do you really understand why water boils?” is deceptive and depicts the population as being less intelligent than they are.

    (ND: I disagree (obviously). Unless you’ve memorized the relationship between altitude and boiling points, or are familiar with various high-altitude cooking instructions, the only way to work through this question is by recognizing how altitude — atmospheric pressure — affects the boiling point of a substance. If you remember that boiling points are not just related to the temperature of a substance, but also to the pressure of the environment it’s in, reasoning will lead you to the right answer. Thus, you DO have to understand why water boils to answer it correctly.)

  7. You realize that the moon’s mass and distance from the earth means (according to Newton) it has an upward gravitational pull on the earth’s oceans that is equal to only about 1/300,000th of the downward pull of the earth’s surface gravity. Try picking up a brick out in your backyard by exerting a force on it equal to 1/300,000th of earth’s surface gravity and see how far it moves. It won’t leave the ground even a tiny bit. The lifting force has to exceed the surface gravity, obviously. The moon’s motion corresponds to the tides (for reasons that go far back in time to the formation of the earth/moon system) but can’t possibly cause them. Galileo was correct in his speculations about the tides.

Leave a Reply

Your email address will not be published. Required fields are marked *