Nobel laureate Daniel Kahneman calls for ‘daisy chain’ of psychology replications
Last Wednesday, Nobel laureate Daniel Kahneman sent an email to a group of a dozen or so psychologists telling them that the credibility of their field was in danger. The recipients all worked on social priming – the study of how subtle unconscious cues can influence our behaviour. It’s an area that has attracted controversy of late, due to failed replications of classic results, the outing of fraudulent researchers, and a more general concern among psychologists about the validity of their field’s results.
Kahneman meant the email as helpful advice, but his wording couldn’t have been stronger: “Your field is now the poster child for doubts about the integrity of psychological research… I believe that you should collectively do something about this mess.” His solution: a “daisy chain” of replications, where laboratories collaborate to check the results of their neighbours, in an open, transparent, and pre-established way.
Kahneman requested that the email be sent to anyone relevant and, presumably because I have written several pieces on this topic, a copy landed in my Inbox on Wednesday. I interviewed Kahneman about it, and my story about his challenge (my description, not his) is now up at Nature News.
Here’s some overmatter that didn’t make it into the piece because of length.
- I quoted Norbert Schwarz in the piece, but he sent over a far lengthier set of comments, which are online in a Google doc.
- John Bargh, whose name will be familiar to readers of this blog, sent me comments that were cut from the piece during editing. They echo Schwarz’s views in emphasising the “extensive” support for priming effects—“If people continue to read just critiques and calls to action, the perception of a replication problem will probably continue to grow. I would urge instead a careful reading of the literature as a whole.” But he is also broadly supportive of Kahneman’s suggestion: “I do not see any harm that could be done if social cognition researchers began to replicate each other’s new findings, which is what Danny called for… I support Danny’s suggestion or some similar mechanism that helps to accomplish this important goal of further establishing the reliability of new effects in social psychology.”
- David Funder, the President-Elect of the SPSP (Society for Personality and Social Psychology) thinks that Kahneman’s email carries a lot of weight with it. “He must be the most prestigious living psychologist. He’s our one and only living Nobel laureate and has a huge amount of respect from just about everybody. He’s just trying to play referee, and telling people to dial down the anger and defensiveness. One of the things that has made me sad about how this has played out is that this should be a scientific discussion and not an argument. For scientists, it should be that the data is the data, and let the chips fall where they may.”
- In his email, and in a phone interview with me, Kahneman said that priming effects are very subtle, and could be undermined by small changes to experimental protocols at the hands of unskilled experimenters. “Replication is intended as cloning but there are mutations and they can be lethal,” he said. This is why his “daisy chain” idea involves people within the priming field replicating each others’ studies. But social psychologist Brian Nosek makes this good point: “People say there is nuance here, but that’s a hypothesis. That’s not true prima facie. I’d love to see collaborations across laboratories and to see if [expertise] really matters.”We’d then learn not just if results are true, but why they turn up in some labs and not others.
- Kahneman mentions recently outed fraudsters in his email – these would include Diederik Stapel from last year, and Dirk Smeesters and Lawrence Sanna this year, all of whom used priming techniques in their experiments. I asked him about this, and the effect on the priming field. “It is a feature of priming studies that they’re surprising; some of them have been quite exuberant in the kind of things they have shown,” he says. “It doesn’t follow that people who are doing priming are committing fraud; it’s more likely that people who are committing fraud are drawn to that field. It’s important not to invert the logic of that.”
Related Topics
Go Further
Animals
- This ‘saber-toothed’ salmon wasn’t quite what we thoughtThis ‘saber-toothed’ salmon wasn’t quite what we thought
- Why this rhino-zebra friendship makes perfect senseWhy this rhino-zebra friendship makes perfect sense
- When did bioluminescence evolve? It’s older than we thought.When did bioluminescence evolve? It’s older than we thought.
- Soy, skim … spider. Are any of these technically milk?Soy, skim … spider. Are any of these technically milk?
- This pristine piece of the Amazon shows nature’s resilienceThis pristine piece of the Amazon shows nature’s resilience
Environment
- This pristine piece of the Amazon shows nature’s resilienceThis pristine piece of the Amazon shows nature’s resilience
- Listen to 30 years of climate change transformed into haunting musicListen to 30 years of climate change transformed into haunting music
- This ancient society tried to stop El Niño—with child sacrificeThis ancient society tried to stop El Niño—with child sacrifice
- U.S. plans to clean its drinking water. What does that mean?U.S. plans to clean its drinking water. What does that mean?
History & Culture
- Meet the original members of the tortured poets departmentMeet the original members of the tortured poets department
- Séances at the White House? Why these first ladies turned to the occultSéances at the White House? Why these first ladies turned to the occult
- Gambling is everywhere now. When is that a problem?Gambling is everywhere now. When is that a problem?
- Beauty is pain—at least it was in 17th-century SpainBeauty is pain—at least it was in 17th-century Spain
- The real spies who inspired ‘The Ministry of Ungentlemanly Warfare’The real spies who inspired ‘The Ministry of Ungentlemanly Warfare’
Science
- Here's how astronomers found one of the rarest phenomenons in spaceHere's how astronomers found one of the rarest phenomenons in space
- Not an extrovert or introvert? There’s a word for that.Not an extrovert or introvert? There’s a word for that.
- NASA has a plan to clean up space junk—but is going green enough?NASA has a plan to clean up space junk—but is going green enough?
- Soy, skim … spider. Are any of these technically milk?Soy, skim … spider. Are any of these technically milk?
Travel
- What it's like to hike the Camino del Mayab in MexicoWhat it's like to hike the Camino del Mayab in Mexico
- Is this small English town Yorkshire's culinary capital?Is this small English town Yorkshire's culinary capital?
- This chef is taking Indian cuisine in a bold new directionThis chef is taking Indian cuisine in a bold new direction