Header image

john hawks weblog

paleoanthropology, genetics and evolution

Photo Credit: Photo by Guillaume Jaillet on Unsplash

What keeps astronomers from publishing their results?

An article in Science by Daniel Clery investigates the mystery of why half the astronomers who are granted telescope time never seem to publish their results: “‘Still working’: Astronomers explain why they don’t publish”.

I’ve often compared anthropology to astronomy in terms of data sharing and data generation. Astronomy is moving toward massive open datasets from enormous sky survey telescopes. But some of the most expensive and largest telescopes apportion time to researchers based upon a proposal system, allocating hours of observation time on instruments that cost hundreds of millions of dollars to build and maintain.

Ferdinando Patat surveyed scientists who were granted observation time between 2006 and 2013 but still haven’t produced any peer-reviewed outputs:

They got a surprisingly high number of responses—80%—and the most common one was, perhaps unsurprisingly, “I am still working on the data.” [Ferdinando] Patat says, “That’s the easiest answer you can give, like when you ask a student why they haven’t submitted their essay on time.” But perhaps they’re not trying to pull a fast one. Patat says other studies have shown an asymptotic curve of publication delay, which takes about 3.5 years to reach 50% of the total number of publications and 10 years to reach 95%.

I’ve heard many people suggest that there is a tradeoff between doing work in a reasonable time and doing work of high quality.

However, there are many reasons why taking a long time tends to degrade the quality of work. How many times have you looked at a draft of a paper from five years ago, and now you can’t remember the details that go into its analyses?

When you remove a publication to a time long after data collection, details on how the data were collected may be lost. Indeed, generating new datasets to compare to the original ones may be impossible.

Most science today is done by teams, and team members move, change jobs, or go on to new projects, all of which can negatively impact the quality of a team product. Especially when team members know that a piece of research is on the slow track, many will not give it the priority in their workflow that would result in the highest quality output.

What is a problem in the astronomy case is that telescope time is viewed as a valuable research goal in itself, setting aside that it is necessary for original analyses:

Patat says you can never get to 100% because it is part of the scientific process that some risky proposals may never produce results. Part of the shortfall he ascribes to the trend throughout science to avoid publishing negative results. “This reflects what may be a growing cultural problem in the community as scientists tend to concentrate on appealing results, especially if they have limited resources, and the need to focus predominantly on projects that promise to increase their visibility,” Patat says. But he also suspects there are some proposals that are not well thought through or are thrown in to show a team is busy. “It’s a perverse system where winning time on its own is seen as important,” he says.

You may think it’s a stretch to compare this to anthropological fieldwork, but there are many, many researchers who view fieldwork as a goal in itself, rather than a means to publishing original research. Many have constructed fieldwork as an enterprise that occurs over two or three weeks a year, very much like telescope observation time is a limited number of nights.