Moving toward preprint reviews at eLife

3 minute read

The journal eLife has adopted a new editorial policy expecting all submitted papers to be first posted to a preprint server like bioRxiv. The policy is announced in an editorial: “Peer Review: Implementing a ‘publish, then review’ model of publishing”. Science is running an interview with eLife editor-in-chief Michael Eisen: “In biology publishing shakeup, eLife will require submissions to be posted as preprints”.

Some scientists will be excited by this news, while others probably think this another of the disasters of 2020. I’m cautiously optimistic.

The eLife editorial reports that nearly 70% of papers submitted to the journal are already first posted as preprints. The new policy will require all submissions to be public preprints. The editorial discusses the possibility of exceptions:

However, we are aware that some potential authors have real concerns about posting preprints, and others are restricted from doing so for various reasons. We would like to take some time to understand and, where possible, mitigate these concerns, especially in areas where adoption of preprints has lagged behind. Therefore, for the next six months, we will give authors of papers under review the option to opt out of posting a preprint, and will instead ask them to explain their reasons.
Once we feel we understand and have addressed concerns about preprinting that are in our power to address, we will move to only review papers that are already available as preprints. It will still be possible for authors to request a waiver of this policy in exceptional circumstances, but we expect these to be rare.

I’ve published several of my best scientific papers with eLife and I watch the journal closely because it is a leader in open access and innovation in scientific publishing. Will other journals follow?

The PLoS organization in October reported on a yearlong trial incorporating bioRxiv comments into their peer review process: “Community Comments Continued: An update on the preprint commenting pilot at PLOS”. In the process, they quantified the kinds of public review the preprints on bioRxiv are getting. The depressing answer is, not much:

As we anticipated, during this pilot the majority of eligible preprints haven’t received any public comments on the bioRxiv platform. Staff facilitating this pilot have observed that, while some preprints receive high quality feedback in the comments, most do not. Also, the timing of preprint commenting has not always lined up with the peer review process, so that preprints have received detailed feedback after an editorial decision had already been made.

There is very little incentive for researchers to provide peer review in the first place. At least when a journal editor makes a request, a researcher knows that someone will read the review.

In genomics, a large fraction of research groups now post preprints either with submission or before submission to a journal. What’s interesting is that I don’t usually notice many changes between these preprints and the eventually published papers. I think that in genomics, researchers can often be confident that the review process will require few changes to their work. Preprints thus are a fairly accurate signal that the work will be published in a journal in a similar form. There have been of course some notable exceptions, and a fair number of preprints are never published in journals.

My experience with preprints has been uneven. Some have gotten great feedback from other researchers, improving the final product. Others haven’t gotten much attention at all. I have not had strongly negative experiences with any preprints. I have done a good amount of pre-publication review of preprints by other researchers. Usually this has taken the form of blogging about preprints and pointing out strengths and weaknesses of the manuscripts. I have also corresponded confidentially with authors to comment on preprints. This kind of e-mail comment is how I have received all the feedback on my preprints from other researchers.

I actually think that the present trend toward adopting preprints without a robust public commenting culture works to the benefit of selective journals like Nature, Science, and Cell. These selective journals curate and validate papers. The more confusing the system is, the more these existing players benefit. And certainly the current situation is confusing.

A vibrant ecosystem of public assessment of manuscripts would provide ways for readers to identify valid research independent of publishers. I expect that this is why eLife is trying to shake up the system. But we are a long way from a system in researchers provide as much effort to assessment as will be needed.