My new story on psychology’s problem with replications
I have a new feature out in Nature looking at two big problems within the field of psychology. First, the field is almost entirely dominated by positive results, while negative ones languish unpublished in personal file drawers. Second, there are few incentives to replicate old results and negative replication attempts face a lengthy gauntlet of obstacles. In the story, I look at why these problems exist and why some psychologists are starting to take them very seriously.
The piece has its origins in an incident that regular readers will already know. In January, Stephane Doyen and colleagues had unsuccessfully tried to repeat a classic experiment where people walk more slowly down a corridor after being unconsciously primed with age-related words. I wrote about their research. Two months later, the man behind the original study – John Bargh of Yale University – wrote a scathing attack on Doyen’s team, me, and the journal that published the study. I responded.
The ensuing discussion opened my eyes to an undercurrent of unrest. Many psychologists came out of the woodwork to mention experiments that were hard to replicate, common practices that they deemed to be dodgy, and a growing willingness to turn a critical eye upon their own field. For every comment that appeared on the blog and Twitter, I’ve got another that was sent confidentially to me via email. This was clearly something worth writing about.
From the outset, I was very clear that this was not going to be a hit piece, against any individual researcher, or psychology as a field, and I’d like to think that the result is a fair assessment. I look at issues such as: conceptual replication (where different but thematically similar experiments are said to replicate each other); the difficulties of repeating studies of abstract concepts; the role of almost magical ‘knacks’; and how tiny, innocuously-made choices that pull statistically significant results from the ether. I also look at the extent to which these problems are unique to psychology, compared to other sciences.
And perhaps most importantly, I look at the triggers that have provoked many psychologists into recognising and discussing these problems, from controversial papers about psychic abilities to high-profile frauds like Diederik Stapel. It is not my contention that such misconduct is common but I think that the field’s cultural norms have created an environment where Stapels could go unnoticed.
But enough background. Do read the piece, and let me know what you think. I might post a bit more about it, depending on reactions.
And finally, many thanks to Brendan Maher for seeing the potential in the story and editing it into shape, and Helen Pearson for taking it over the finishing line.
Update:
I’m going to collect some other good resources on this issue:
- An entire special issue on the topic from The Psychologist
- Psychology’s bold initiative – great piece from Siri Carpenter about Brian Nosek and his Reproducibility Project
- Navel gazing on steroids in social and personality science – Brent Roberts summarises some of the recent events, including some I didn’t touch on in the piece
- Bias in psychology – Razib Khan riffs off my piece.
Related Topics
Go Further
Animals
- Octopuses have a lot of secrets. Can you guess 8 of them?
- Animals
- Feature
Octopuses have a lot of secrets. Can you guess 8 of them? - This biologist and her rescue dog help protect bears in the AndesThis biologist and her rescue dog help protect bears in the Andes
- An octopus invited this writer into her tank—and her secret worldAn octopus invited this writer into her tank—and her secret world
- Peace-loving bonobos are more aggressive than we thoughtPeace-loving bonobos are more aggressive than we thought
Environment
- Listen to 30 years of climate change transformed into haunting musicListen to 30 years of climate change transformed into haunting music
- This ancient society tried to stop El Niño—with child sacrificeThis ancient society tried to stop El Niño—with child sacrifice
- U.S. plans to clean its drinking water. What does that mean?U.S. plans to clean its drinking water. What does that mean?
- Food systems: supporting the triangle of food security, Video Story
- Paid Content
Food systems: supporting the triangle of food security - Will we ever solve the mystery of the Mima mounds?Will we ever solve the mystery of the Mima mounds?
History & Culture
- Strange clues in a Maya temple reveal a fiery political dramaStrange clues in a Maya temple reveal a fiery political drama
- How technology is revealing secrets in these ancient scrollsHow technology is revealing secrets in these ancient scrolls
- Pilgrimages aren’t just spiritual anymore. They’re a workout.Pilgrimages aren’t just spiritual anymore. They’re a workout.
- This ancient society tried to stop El Niño—with child sacrificeThis ancient society tried to stop El Niño—with child sacrifice
- This ancient cure was just revived in a lab. Does it work?This ancient cure was just revived in a lab. Does it work?
Science
- The unexpected health benefits of Ozempic and MounjaroThe unexpected health benefits of Ozempic and Mounjaro
- Do you have an inner monologue? Here’s what it reveals about you.Do you have an inner monologue? Here’s what it reveals about you.
- Jupiter’s volcanic moon Io has been erupting for billions of yearsJupiter’s volcanic moon Io has been erupting for billions of years
- This 80-foot-long sea monster was the killer whale of its timeThis 80-foot-long sea monster was the killer whale of its time
Travel
- How to plan an epic summer trip to a national parkHow to plan an epic summer trip to a national park
- This town is the Alps' first European Capital of CultureThis town is the Alps' first European Capital of Culture
- This royal city lies in the shadow of Kuala LumpurThis royal city lies in the shadow of Kuala Lumpur
- This author tells the story of crypto-trading Mongolian nomadsThis author tells the story of crypto-trading Mongolian nomads