Misuse of statistics15 Oct 2016
This essay should be required reading for graduate students: “The problem with p-values”. David Colquhoun writes extensively about science and statistics, and in this essay he brings out many of the biggest misconceptions that drive poor conclusions in scientific practice.
Even quite respectable sources will tell you that the p-value is the probability that your observations occurred by chance. And that is plain wrong.
Paleoanthropology is one field in which papers that point out poor use of statistics are publishable. The primary fossil data are very sparse, and we work hard to establish what little we can say with confidence. There are generally scientists willing to criticize statistically misleading attempts to answer the unanswerable.
Personally, I’ve spent a good amount of time looking into the basic statistical underpinnings of human evolution datasets. For example, my paper “How much can cladistics tell us about early hominid relationships?” showed that the datasets of most hominin species are simply not big enough to yield confident conclusions about how they are related to each other. A later paper, “No brain expansion in Australopithecus boisei”, worked through statistical issues with time-series data on which paleoanthropologists have often based conclusions about trends in morphological features over time.
My all-time favorite paper outlining statistical problems in human evolution research is by Richard Smith, “Biology and body size in human evolution: statistical inference misapplied.” Smith shows a systematic problem with the most common comparisons of ancient human relatives. Now, twenty years after that paper was first published, most papers that consider body masses of fossil hominins still get this wrong.
That’s a well-worn tale in science. Pointing out statistical errors is sisyphean. Many recent papers in human evolution reflect poor statistical practice. And a good number of “classic” results are based on datasets that today would be statistically doubtful. Science is self-correcting, but it is going to take some hard work to get this stuff straight.
Hawks, J. (2004). How much can cladistics tell us about early hominid relationships?. American journal of physical anthropology, 125(3), 207-219. doi:10.1002/ajpa.10280
Hawks, J. (2011). No brain expansion in Australopithecus boisei. American journal of physical anthropology, 146(2), 155-160. doi:10.1002/ajpa.21420
Smith, R. J. (1996). Biology and body size in human evolution: statistical inference misapplied. Current Anthropology, 37(3), 451-481. JSTOR