On the importance of saying you're wrong

Lior Pachter writes this week on his blog about the reactions and commentary around a post-publication peer review exercise he conducted on a 11-year-old paper. In the process, he reflects on some of the problems that attend the frank public conversation about weaknesses and strengths of scientific work: “I was wrong”.

Earlier in this post I admitted to being wrong. I have been wrong many times. Even though I’ve admitted some of my mistakes on this blog and elsewhere in talks, I would like to joke that I’m not going to make it easy for you to find other flaws in my work. That would be a terrible mistake. Saying “I was wrong” is important for science and essential for scientists. Without it people lose trust in both.
I have been particularly concerned with a lack of “I was wrong” in genomics. Unfortunately, there is a culture that has developed among “leaders” in the field where the three words admitting error or wrongdoing are taboo.

He discusses recent examples including the snafu induced by the ENCODE Consortium’s insistence that 80% of the human genome is “functional”. He goes on to discuss the importance of public critique of the professional behavior of scientists, with a quote that deserves sharing:

I therefore believe it is not only acceptable but imperative to critique the professional behavior of persons who are scientists. I also think that doing so will help eliminate the problematic devil–saint dichotomy that persists with the current system. Having developed a culture in which personal criticism is outlawed in scientific conversations while only science is fair fodder for public discourse, we now have a situation where scientists are all presumed to be living Gods, or else serious criminals to be outlawed and banished from the scientific community. Acknowledging that there ought to be a grey zone, and developing a healthy culture where critique of all aspects of science and scientists is possible and encouraged would relieve a lot of pressure within the current system. It would also be more fair and just.

The study he describes in the post, and the attendant experiment in examining p-values for certain kinds of genomic questions, are interesting and worth reading. As many have pointed out, it is unfortunate that so few pieces of scientific work can achieve such broad interest in the form of public discussion.

I also think it’s worth sharing an excerpt from a comment at the site from Claudiu Bandea:

“The traditional, closed peer-review system and the conventional ‘civility’ associated with the science enterprise, have allowed, if not encouraged, people to prosper by misrepresenting facts and overhyping their work (at the expense of science and their colleagues), without the *fear* of open and explicit exposure.
And, unfortunately, in order to be able to compete in such a corrupt environment, many of their peers had little choice but to lower their ethical and scientific standards and join this unproductive and reckless competition; and, by doing so, all of them (even the most reckless ones, who often display embarrassing CVs inflated with pompous titles and rewards) have become victims of the system.

When I hear people talk about being unwilling to engage in debate except in the “proper” peer-reviewed forums, this is exactly what I assume they mean.