Let’s make replicability a priority in human evolution research09 Jan 2017
Research on human evolution may have the worst history of data access for any field of science funded by the National Science Foundation. NSF has spent millions of dollars over the last twenty years on results that cannot be replicated. This is bad science.
I have been proud to have NSF select me to review a number of proposals over the years. But I am frustrated that my comments on proposals pointing to problems with data access plans have had no visible effects. Those of us who review proposals should use our reviews to turn around the trend of bad data access.
Before going further, I want to say that I recognize the hard work of NSF program officers and panels. Many of the field projects currently funded by NSF in biological anthropology and archaeology have established data access policies that allow independent scientists to see and replicate results. NSF has additionally directed funding toward databases, like MorphoSource, that allow researchers to archive data and share it with other researchers or the public. That’s good science.
With many scientists doing a good job, and every proposal now requiring a data access plan, it is frustrating to see some projects providing only lip service to data access.
Some research groups have resorted to chicanery. One project was funded to provide a database for all vertebrate specimens recovered at their field site, complete with scaled photographs. But hominin fossil material was accompanied only by tiny unscaled thumbnail images.
One project became notorious for publishing research in which standard measurements were noted as “present” or “absent” instead of providing the measurement values. All while acknowledging NSF funding.
Such examples are childish. Let’s stop funding projects that fail basic data reporting. We won’t be losing anything valuable. We’ll be gaining the opportunity to fund work that others can build upon.
Of course many anthropologists collect data within countries where government policy may restrict the open dissemination of original data. Some may say it would be unfair for NSF to discriminate against applicants who work in these circumstances.
But asking for basic reporting is not discrimination. NSF-funded researchers should facilitate reasonable access to their data. That doesn’t have to mean open access to the public. Good science demands the potential of replication, with data responsibly archived and clear instructions on how bona fide researchers can obtain permission to replicate results.
Recently I have heard that certain researchers permit other scholars to examine original fossil material, collected using NSF funds, only after they sit with casts before them and agree in advance upon the conclusions they will reach. If this practice is actually happening, it has to stop.
Fossils are national and world heritage objects, and access to these treasures is a privilege. Access to U.S. public funding is also a privilege, one that in today’s budget environment only a tiny minority of scholars will ever attain. I have been honored over the years to serve NSF in reviewing proposals, though I have never received funding myself. Like many, I volunteer sharing the hope that funding will be directed toward the best projects with the broadest impacts on the science. That cannot happen if data access is arbitrary, used to reward friends and punish enemies, only allowed when conclusions are preordained.
I call upon panelists and reviewers in this funding cycle to make data access a priority when you review research proposals. When you see a proposal with a strong data access plan and a record of demonstrable success, fund it. When you see a proposal from researchers with poor track records of data access, please just say no.
U.S. federal funding for human evolution research is precarious and could be taken away at any time. We can best sustain it by showing that we are serious about promoting the best, most replicable science.
I published this essay previously on Facebook.