Peerless critiques

Nature last week posted an open access editorial, “Response required”, on the need for authors of high-profile papers to engage with online commentary on blogs and other forums. The arsenate bacteria story was the point of departure for the editorial:

Formal peer review does give criticized authors time to think critically and carefully, and it is a good way to filter out rubbish. But in this case, much of the criticism was already coming from the researchers' peers. And it should be remembered that peer review as conducted by journals is itself full of differing opinions, and is not the only way to crystallize truth from such disputes. In this instance, a prompt and explicitly provisional response from the authors would have been a better approach, particularly given the way they encouraged the original attention.

The comment thread on the editorial has a range of perspectives, some defending researchers who “clam up”, others trashing them.

Our expectations are that “extraordinary claims require extraordinary evidence”. Our system of scientific message dissemination is tolerant of incomplete or misleading research. Heck, it’s even tolerant of pure wackos – otherwise, the “scientists” who appear on Ancient Aliens would cause a weekly crisis. What our system doesn’t tolerate well is “elite” organizations like NASA sending the science equivalent of spam messages. At the History Channel, nobody expects that the scientists are running the show. From organizations like NASA, we expect more.

A common strain is that the whole thing is that hype is the fault of those who write the press release, not the researchers. I can’t agree with that. We have no way of stopping hyped-up science unless we make researchers pay a price for their complicity. The public attention span is limited, drawn most potently by messages that are short, punchy and surprising. Most scientific research cannot be reduced to these qualities.