Should people lay off social psychologists with unreplicable ideas?

Pardis Sabeti has written an op/ed for the Boston Globe addressing the scientific casualties of the “replication crisis” in social psychology: “For better science, call off the revolutionaries”.

After naming a number of psychologists whose work has come under criticism for overhyped claims and underpowered samples, she decries the level of public criticism that these scientists have received:

Good science requires a spirit of collaboration, not domination. The debate in social psychology involves some essential criticism of past scientific practice, but revolutions can also lead to a bandwagon effect, in which bullies pile on and bystanders fearfully turn a blind eye. Especially as more disagreements among researchers surface in social media rather than professional publications, there is an insidious temptation to mistake being critical for being right, and to subordinate humility and decency to a “gloating sense of ‘gotcha,’” as the journal Nature put it.
There is a better way forward: through evolution, not revolution.

Sabeti contrasts the current situation in social psychology with genomics. In her account, much work in human genetics came under similar criticism in the early 2000s for relying upon underpowered samples, finding gene-trait associations that didn’t satisfy rigorous statistical requirements.

According to Sabeti, human genetics did not take the path of strong public criticism of underpowered studies, or of vilification of the researchers trying to find gene-trait associations with too little data. Instead, human geneticists built upon the imperfect results, revising them as samples got better and technology accelerated.

But there are many additional reasons why human genetics in the early 2000s was different from social psychology today.

The Human Genome Project during the late 1990s presented an enormous series of challenges to mainstream academic human geneticists. Scientists in government and in universities, led at first by James Watson and later by Francis Collins, rose to collective action, racing with Craig Venter’s private Celera venture to complete a draft of the human genome.

In doing so, they established a pattern of work that was unprecedented in biological science—in the process, building new expectations and protocols for sharing data. These scientists forged a collective understanding of the “big challenges” facing human genetics in the next decade. This included the critical need greatly increase knowledge of variation across the genome, to bring additional populations into human genetic samples, and to increase understanding of the genetic architecture of traits, especially disease traits.

That project established a hierarchy of scientists recognized as leaders, embedded within or experienced working with NIH, DOE, and other major funders. Some became directors of newly-founded major research institutes, among the first creations of the burgeoning philanthropic and government funding that moved into genome sequencing.

Few fields of science have ever seen the increase in money, power, and prestige poured into human genetics during the early 2000s. It’s not accurate to say this was a world free of conflict. Many of the disagreements within the field were acrimonious, and theoretical paradigms like common disease/common variant provoked strong debates. But none of these really compared to the rivalry between the public HGP and Celera during the 1990s. Meanwhile, prestige within the field came from directing and coordinating the Big Science efforts. That enabled project leaders to cultivate early career talent and select the most promising ideas from young people.

By contrast, social psychology has relatively little funding and no “Big Science” project. The competition for prestige and attention is very different than in human genetics. Psychologists write popular books, pen op/eds, and give Ted talks.

It’s like the Wild West, compared to human genetics. And where genetics was in the middle of a huge technological shift enabling vastly larger samples and more data for less money, samples in social psychology have been getting more expensive over time, not cheaper.

It will take a shock to the academic system to change the culture in such fields, toward pre-registered studies, vastly larger samples, and open data approaches.