My new story on psychology’s problem with replications
I have a new feature out in Nature looking at two big problems within the field of psychology. First, the field is almost entirely dominated by positive results, while negative ones languish unpublished in personal file drawers. Second, there are few incentives to replicate old results and negative replication attempts face a lengthy gauntlet of obstacles. In the story, I look at why these problems exist and why some psychologists are starting to take them very seriously.
The piece has its origins in an incident that regular readers will already know. In January, Stephane Doyen and colleagues had unsuccessfully tried to repeat a classic experiment where people walk more slowly down a corridor after being unconsciously primed with age-related words. I wrote about their research. Two months later, the man behind the original study – John Bargh of Yale University – wrote a scathing attack on Doyen’s team, me, and the journal that published the study. I responded.
The ensuing discussion opened my eyes to an undercurrent of unrest. Many psychologists came out of the woodwork to mention experiments that were hard to replicate, common practices that they deemed to be dodgy, and a growing willingness to turn a critical eye upon their own field. For every comment that appeared on the blog and Twitter, I’ve got another that was sent confidentially to me via email. This was clearly something worth writing about.
From the outset, I was very clear that this was not going to be a hit piece, against any individual researcher, or psychology as a field, and I’d like to think that the result is a fair assessment. I look at issues such as: conceptual replication (where different but thematically similar experiments are said to replicate each other); the difficulties of repeating studies of abstract concepts; the role of almost magical ‘knacks’; and how tiny, innocuously-made choices that pull statistically significant results from the ether. I also look at the extent to which these problems are unique to psychology, compared to other sciences.
And perhaps most importantly, I look at the triggers that have provoked many psychologists into recognising and discussing these problems, from controversial papers about psychic abilities to high-profile frauds like Diederik Stapel. It is not my contention that such misconduct is common but I think that the field’s cultural norms have created an environment where Stapels could go unnoticed.
But enough background. Do read the piece, and let me know what you think. I might post a bit more about it, depending on reactions.
And finally, many thanks to Brendan Maher for seeing the potential in the story and editing it into shape, and Helen Pearson for taking it over the finishing line.
Update:
I’m going to collect some other good resources on this issue:
- An entire special issue on the topic from The Psychologist
- Psychology’s bold initiative – great piece from Siri Carpenter about Brian Nosek and his Reproducibility Project
- Navel gazing on steroids in social and personality science – Brent Roberts summarises some of the recent events, including some I didn’t touch on in the piece
- Bias in psychology – Razib Khan riffs off my piece.
Related Topics
Go Further
Animals
- These 'trash fish' are among Earth's most primitive animalsThese 'trash fish' are among Earth's most primitive animals
- These photos are works of art—and the artists are bugsThese photos are works of art—and the artists are bugs
- The epic migration of a 6-foot long, 200-pound catfishThe epic migration of a 6-foot long, 200-pound catfish
- Frans de Waal, biologist who studied animal emotion, dies at 75Frans de Waal, biologist who studied animal emotion, dies at 75
Environment
- Are synthetic diamonds really better for the planet? The answer isn't clear-cut.Are synthetic diamonds really better for the planet? The answer isn't clear-cut.
- This year's cherry blossom peak bloom was a warning signThis year's cherry blossom peak bloom was a warning sign
- The U.S. just announced an asbestos ban. What took so long?The U.S. just announced an asbestos ban. What took so long?
- The most dangerous job? Inside the world of underwater weldersThe most dangerous job? Inside the world of underwater welders
- The harrowing flight that wild whooping cranes make to surviveThe harrowing flight that wild whooping cranes make to survive
History & Culture
- Meet the powerful yokai that inspired the demon king in ‘Demon Slayer’Meet the powerful yokai that inspired the demon king in ‘Demon Slayer’
- A surprising must-wear for European monarchs? Weasels.A surprising must-wear for European monarchs? Weasels.
- Meet the woman who made Polaroid into a cultural iconMeet the woman who made Polaroid into a cultural icon
- Inside the observatory that birthed modern astrophysicsInside the observatory that birthed modern astrophysics
Science
- LED light treatments for skin are trendy—but do they actually work?LED light treatments for skin are trendy—but do they actually work?
- NASA smashed an asteroid. The debris could hit Mars.NASA smashed an asteroid. The debris could hit Mars.
- Humans really can have superpowers—scientists are studying themHumans really can have superpowers—scientists are studying them
- Why engineers are concerned about aging infrastructureWhy engineers are concerned about aging infrastructure
Travel
- 2024 will be huge for astrotourism—here’s how to plan your trip2024 will be huge for astrotourism—here’s how to plan your trip
- Play and stay in the mountains of eastern Nevada
- Paid Content
Play and stay in the mountains of eastern Nevada - This couple quit the city to grow wasabi in Japan's mountainsThis couple quit the city to grow wasabi in Japan's mountains