The daily business of science journalism includes getting independent comments on new studies, and (in my opinion) providing those comments is one of the most important ways in which scientists interact with the media.
But from talking to scientists on Twitter, I know that there’s a lot of nervousness about giving comments to journalists. And when I send papers out to people for comments, I often get replies that say, “Sure, but what do you want me to say?”
A straw poll, conducted this morning on Twitter, suggested that people would value a guide on doing this. So, here’s what I am looking for. (To clarify, this isn’t me asking about your own work. It’s me asking you to comment on someone else’s work.)
Let’s start with some assumptions.
These may not be true for all journalists, but they’ll be Standard Operating Procedure for the good ones.
A) I have read the paper that I sent you and understand it (or I’ve talked to the scientists in question and they’ve explained it to me). So you don’t need to explain its contents back to me.
B) I’m coming to you because you are an expert in this field and you know lots of stuff about it that I don’t know. I want you to use that expertise to help me put the research into context for my readers, and to help me point out any flaws and strengths.
C) I genuinely want to know what you make of the paper. I am not just trying to fill my story with a random cutaway quote to make it look like I did my job and asked around.
D) I’m not here to present people with the totality of your views, so what you say will almost certainly end up getting cut and distilled. BUT, I won’t do that in a way that misquotes or misrepresents you. If you say, “I’m fascinated by this approach but I think it has serious flaws”, I won’t cut that to “I’m fascinated”. I’m a journalist; I’m not making a movie poster.
E) All the tips below apply to situations where I email you a paper and ask for comments. If we’re chatting on the phone, it’s my job to guide you through all of this, but it will obviously take less time for both of us if you know what I’m after. And I’m talking about written interviews. Some of these will apply to TV and radio too, but those have very different constraints.
So, here’s what I would find useful:
1) Weaknesses. The most important things you can tell me about a study are its weaknesses. Are there inaccuracies in the paper? Statistical failings? Do you think the conclusions don’t hold water? The last thing I want to do is to credulously cover a weak study. But I don’t work in your field and my bullsh*t detector is probably less finely calibrated than yours. So I’m basically relying on you to help me not mislead my readers. Maybe your comments will persuade me to drop a story because it’s just that bad. Maybe your comments will help me to confront an editor and say: “We shouldn’t cover this story that you seem so insistent on. Look: all these scientists think it’s bunk.”
2) Strengths. But hey, it’s not all doom and gloom! If you’re excited, I want to hear that! Go, science, etc.! But also, tell me the reasons for that excitement. Did they get an unprecedentedly big data set? Some cool new method? An unusual model organism? Innovative technique?
3) Your reaction. When you read the paper, how did it make you feel? Were you excited? Impressed? Overwhelmed by a deep existential malaise?
4) The past. The paper will probably have a paragraph that crushes decades of earlier work. You will know all of that; I won’t have had time to read all those earlier papers. So tell me: How does this new discovery fit with what has come before? Is this based on a radical new approach? A long slog? Something that people in the field have been anticipating? Is it just reinventing the wheel?
5) The present. Have other people found similar things? Contradictory things? Is this one of many such studies, or something truly original? If this is, say, a new approach to fighting malaria, how does it compare to all the other approaches people are investigating?
6) The future. So, new discovery. Great. But what does it mean? Does it change what we knew about X? Does it open up new avenues for investigating Y? Will it lead to treatments or diagnostic tools for Z?
7) Detail. Opinions may differ on this, but I like detail and specifics. People sometimes send me quotes that are paragraphs long and “This is probably much more than you need”. That’s true, but I’d rather know all that stuff and have to condense it into something I can use, than to only have something boring, vanilla, and non-descriptive (see the list below).
8) Simple language, in some cases. Look, I know I’m asking a lot here, and it’s a bit much to expect you to lay out all the strength, weaknesses and context of a study for me and have to worry about jargon while you’re doing it. (Could you also rub my shoulders while you’re at it? Thanks ever so.) Just bear in mind that if something is riddled with jargon, I can paraphrase it but I can’t really quote it. That’s a little riskier for you, because maybe I might inadvertently misinterpret something you say. It’s also less good for me. I want to put your words in quote marks because it can really brighten up a piece.
Note that a lot of this boils down to you telling me something interesting that I couldn’t have predicted. That’s why, when people ask me, “Do you have any specific questions?” the answer is often, “No.” What you have to tell me—what springs into your head—is probably going to be far more interesting that anything I’m expecting you to tell me. Hence, any questions I have will be really broad like, “What does this mean?” or “Do you buy it?” or “How does this fit with other stuff?” or “Science me up, nerd.”
Update: I love Tom Stafford’s extra tip of “Don’t be afraid to tell me what the real story is.” Note that this is different to simply summarising the paper.
Now, here’s what I don’t find useful:
1) A summary of what the paper showed. Around half of comments start with this. I don’t need it. I already know what the paper showed, or will have talked to someone else who explained it.
2) Boilerplate adjectives. Please don’t say “This study is interesting…” when you actually mean “dubious” or “boring”.
3) And on that note, the world’s most banal quote is: “This research is interesting but more work needs to be done”. It’s everywhere. It had invaded science stories like some linguistic cane toad. Of course, more research needs to be done. Otherwise, y’know, science would stop. But what research? What needs to be done? If you were doing that research, what experiments would you do? And if by “More work needs to be done” you really mean, “…because this impossibly flawed study tells us nowt”, then say that. Other banal quotes include, “We welcome any research that takes us further down the road towards [hand-wavy goal]” or “This adds to our understanding of [thingy]”.
4) Publication politics. “I don’t know why this paper was published in Nature/Science/FEBS Letters” and other such comments are (usually) not useful. My readers don’t really know, or care about, publication hierarchies. “This paper should never have been published” can be useful for indicating strength of opinion, but I’d always want to know specifics about why. Isolated outrage makes for fun quotes, but not informative quotes.
5) Citation politics. “The authors should have cited this paper instead of that paper.” Again, if an entire body of relevant work has been ignored, then let’s talk about that. But I’m not that bothered about whether reference 55 is the wrong reference 55.
And finally, a note on going off-the record.
Going off-the-record isn’t really a formal, enshrined, binding thing, but if you send me off-the-record comments, I won’t use them. However, my soul will ache when I see “This is off-the-record” followed by a long list of flaws and weaknesses and then “And now on-the-record” followed by something banal.
I get it. If you criticise a study, you risk angering colleagues who work in your field—the same people who you meet at conferences and review your papers. I’m not unsympathetic to that. But as I said, critical comments are probably the most useful variety that we get. You’re in a better position to criticise than I am. And it will probably carry more weight for a reader to see those words coming from the mouth of an expert in the field, than from some journalist. Critical comments do carry personal risk, but they also help us to fight credulous and uncritical science reporting.
Fellow journalists may totally disagree with any and all of this, in which case, have your say in the comments.