My new story on psychology’s problem with replications
I have a new feature out in Nature looking at two big problems within the field of psychology. First, the field is almost entirely dominated by positive results, while negative ones languish unpublished in personal file drawers. Second, there are few incentives to replicate old results and negative replication attempts face a lengthy gauntlet of obstacles. In the story, I look at why these problems exist and why some psychologists are starting to take them very seriously.
The piece has its origins in an incident that regular readers will already know. In January, Stephane Doyen and colleagues had unsuccessfully tried to repeat a classic experiment where people walk more slowly down a corridor after being unconsciously primed with age-related words. I wrote about their research. Two months later, the man behind the original study – John Bargh of Yale University – wrote a scathing attack on Doyen’s team, me, and the journal that published the study. I responded.
The ensuing discussion opened my eyes to an undercurrent of unrest. Many psychologists came out of the woodwork to mention experiments that were hard to replicate, common practices that they deemed to be dodgy, and a growing willingness to turn a critical eye upon their own field. For every comment that appeared on the blog and Twitter, I’ve got another that was sent confidentially to me via email. This was clearly something worth writing about.
From the outset, I was very clear that this was not going to be a hit piece, against any individual researcher, or psychology as a field, and I’d like to think that the result is a fair assessment. I look at issues such as: conceptual replication (where different but thematically similar experiments are said to replicate each other); the difficulties of repeating studies of abstract concepts; the role of almost magical ‘knacks’; and how tiny, innocuously-made choices that pull statistically significant results from the ether. I also look at the extent to which these problems are unique to psychology, compared to other sciences.
And perhaps most importantly, I look at the triggers that have provoked many psychologists into recognising and discussing these problems, from controversial papers about psychic abilities to high-profile frauds like Diederik Stapel. It is not my contention that such misconduct is common but I think that the field’s cultural norms have created an environment where Stapels could go unnoticed.
But enough background. Do read the piece, and let me know what you think. I might post a bit more about it, depending on reactions.
And finally, many thanks to Brendan Maher for seeing the potential in the story and editing it into shape, and Helen Pearson for taking it over the finishing line.
Update:
I’m going to collect some other good resources on this issue:
- An entire special issue on the topic from The Psychologist
- Psychology’s bold initiative – great piece from Siri Carpenter about Brian Nosek and his Reproducibility Project
- Navel gazing on steroids in social and personality science – Brent Roberts summarises some of the recent events, including some I didn’t touch on in the piece
- Bias in psychology – Razib Khan riffs off my piece.
Related Topics
Go Further
Animals
- How scientists are piecing together a sperm whale ‘alphabet’How scientists are piecing together a sperm whale ‘alphabet’
- Orangutan seen using plants to heal wound for first timeOrangutan seen using plants to heal wound for first time
- What La Palma's 'lava tubes' tell us about life on other planetsWhat La Palma's 'lava tubes' tell us about life on other planets
- This fungus turns cicadas into zombies who procreate—then dieThis fungus turns cicadas into zombies who procreate—then die
Environment
- This floating flower is beautiful—but it's wreaking havoc on NigeriaThis floating flower is beautiful—but it's wreaking havoc on Nigeria
- What the Aral Sea might teach us about life after disasterWhat the Aral Sea might teach us about life after disaster
- What La Palma's 'lava tubes' tell us about life on other planetsWhat La Palma's 'lava tubes' tell us about life on other planets
- How fungi form ‘fairy rings’ and inspire superstitionsHow fungi form ‘fairy rings’ and inspire superstitions
- Your favorite foods may not taste the same in the future. Here's why.Your favorite foods may not taste the same in the future. Here's why.
- Are the Great Lakes the key to solving America’s emissions conundrum?Are the Great Lakes the key to solving America’s emissions conundrum?
History & Culture
- These were the real rules of courtship in the ‘Bridgerton’ eraThese were the real rules of courtship in the ‘Bridgerton’ era
- A short history of the Met Gala and its iconic looksA short history of the Met Gala and its iconic looks
- Meet the ruthless king who unified the Kingdom of Hawai'iMeet the ruthless king who unified the Kingdom of Hawai'i
- Hawaii's Lei Day is about so much more than flowersHawaii's Lei Day is about so much more than flowers
Science
- Why ovaries are so crucial to women’s health and longevityWhy ovaries are so crucial to women’s health and longevity
- Orangutan seen using plants to heal wound for first timeOrangutan seen using plants to heal wound for first time
Travel
- Why this unlikely UK destination should be on your radarWhy this unlikely UK destination should be on your radar