How long ago did Neandertals and Denisovans part ways?

16 minute read

We have learned an immense amount about Neandertal population history from their genomes. But many old questions and some new ones remain unanswered.

Among the most basic: How long ago did Neandertal populations become separate from other populations, including Denisovans and ancestral Africans?

Reconstruction of the Neandertal skeleton, from the Neandertal Museum

In early August, Proceedings of the National Academy of Sciences published a paper by Alan Rogers, Ryan Bohlender and Chad Huff that gave a startling new perspective on the origin of Neandertal and Denisovan populations: “Early history of Neanderthals and Denisovans”. At the time, I wrote a perspective piece on the research article, focusing on the implications for rapid dispersal of the ancestral Neandertal-Denisovan population: “Neanderthals and Denisovans as biological invaders.” According to Rogers, Bohlender, and Huff, the Neandertal-Denisovan colonization of Eurasia was a fast expansion and dispersal of a small founder population. It appears to have been an early Middle Pleistocene version of the later colonization of Eurasia by modern humans.

Describing this parallel was easy, but coming to a full understanding of the implications is not. If a model like Rogers, Bohlender, and Huff’s is close to reality, then we will need to radically change some of the ways we look at the fossil and archaeological records.

I’ll describe some of the ways I think this will shake out in several posts over the next few weeks. First, I want to look carefully at the strength of the evidence presented by Rogers and colleagues.

Nothing about interpreting ancient DNA is easy, and I don’t think our current “standard” approaches are adequate to capture the complexity of human prehistory. Most of these interpretations are attempts to fit a model to some statistical summary of the data. By showing that some combinations of parameters fit the data much better than others, it is sometimes possible to reject hypotheses about past populations.

But models are inevitably oversimplifications, and sometimes adding more complexity can resurrect hypotheses that seem inconsistent with simpler models. I’ll go through the model used by Rogers and colleagues, and then point out some of the things that it has omitted.

How it was done

Rogers, Bohlender, and Huff examined the pattern of shared derived mutations in four genomes: an African modern human, a Eurasian modern human, the high-coverage Altai Neandertal genome, and the Denisovan genome. These genomes share different mutations with each other: some are shared between the Neandertal and the Eurasian modern humans, some are shared between the Neandertal and Denisovan, some are shared by three different genomes, and so on.

Here are the data from Rogers and colleagues:

Rogers site frequency results
Site frequency pattern from figure 2a of Rogers and colleagues (2017a). I've added labels indicating which pairs and trios of genomes are which. African and Eurasian modern human genomes share the most derived alleles with each other, Neandertals and Denisovans share many fewer, while all modern and archaic pairs and trios share relatively few derived alleles. However, the modern Eurasian and Neandertal genomes share slightly more than the other combinations of modern and archaic.

In around a third of cases, the two modern humans share the derived mutation. These shared mutations reflect the common shared heritage of these people from Africa, prior to 100,000 years ago.

The Neandertals and Denisovans share fewer mutations with the modern humans; each combination of modern and archaic genomes account for around three or four percent of all the shared mutations. Mostly, the mutations shared by Neandertals or Denisovans and modern humans come from the common origin of all these populations in Africa long before 500,000 years ago. The Neandertal and Eurasian genomes share a small proportion more with each other than the Neandertals and Africans do, and this small proportion reflects the introgression of Neandertal genes into modern human populations.

Shared mutations between the Neandertal and Denisovan genomes account for around 20 percent of the total number of shared mutations from any of the samples. That 20 percent comes from the shared common ancestral population that gave rise to these two archaic populations. Rogers, Bohlender, and Huff wanted to find out how large this ancestral population may have been, and how long it existed before it separated into Neandertal and Denisovan branches.

One innovation of this approach is that it uses more information from the data than the usual method, the “D-statistic” or “ABBA-BABA” method of examining mixture from ancient genomes. With the usual method, researchers are looking at models with mixture and introgression of populations that historically were separated by isolation and genetic drift. By fitting these models, they are trying to estimate the proportion of admixture or introgression, and also measuring the original level of difference between the populations.

Applying more information from the same samples allows Rogers, Bohlender, and Huff to potentially look at richer historical population models. They chose to consider the question of how the Neandertal and Denisovan populations initially became different from each other. For this purpose, they formed a population model that encompasses the separation of a Neandertal-Denisovan ancestral population from the ancestral population of African modern humans, the than the simple mixture and introgression models

Population model from Rogers et al. 2017
Figure 1 from Rogers et al., (2017). Original caption: "Fig. 1. (A) Population tree representing an African population, X; a Eurasian population, Y; Neanderthals, N; and Denisovans, D. The model involves admixture, mN; time parameters, Ti; and population sizes, Ni. (B) Population tree with embedded gene tree. A mutation on the solid red branch would generate site pattern yn (shown in red at the base of the tree). One on the solid blue branch would generate ynd. Mutations on the dashed black branches would be ignored. “0” and “1” represent the ancestral and derived alleles."

Although they can look at more complex models than some other approaches, there is still a limit to what can be discovered from four genomes. When the model involves many more parameters, many different combinations of those parameters may be found to fit the data.

Rogers, Bohlender, and Huff looked at each possible pattern of shared mutations among the four genomes, and this includes ten possible combinations. Each of them accounts for a fraction of the total number of shared mutations, giving ten values for the model to fit. The model shown above includes nine parameters—the effective sizes of four of the ancestral populations, the proportion and timing of introgression from Neandertal into Eurasian populations, and the times of separation of three of the populations. Trying to include more parameters would result in many different parameter combinations being equally good fits to the model—the problem of overfitting.

So this model does not include every aspect of population history that might have been important to the ancestors of the four genomes. It doesn’t include any introgression from ancestral African modern humans into the Neandertals, for example – such as the mixture that must have given rise to the similarity between modern and Neandertal mtDNA. It also doesn’t include introgression into Denisovans from Neandertals or from a “hyperarchaic” ghost population, both of which have been inferred from other kinds of comparisons. These patterns of mixture might throw off the model.

Each of these phenomena might affect the proportion of shared mutations found in both the Neandertal and Denisovan genomes, and that’s potentially important considering that the time of separation of Neandertals and Denisovans is such an important new conclusion.

The results of Rogers and colleagues’ analysis

What is so interesting about Rogers, Bohlender, and Huff’s conclusions is that they find the separation time between Neandertals and Denisovans to be very close after the separation of the Neandertal-Denisovan ancestral population from ancestral Africans:

Rogers and colleagues Neandertal-Denisovan timeline figure
Model for relationships of ancestral Neandertals, Denisovans, and ancestral African modern humans, according to Rogers and colleagues (2017). This model focuses on timeline and does not portray differences in effective population size.

This figure doesn’t encompass every detail of the model investigated by Rogers, Bohlender, and Huff. Probably most important, I have not depicted the estimates of ancestral effective population sizes for each of these populations. And I haven’t included the proportion of admixture that they estimate between Neandertal and ancestral Eurasian populations. In this figure, I’ve only focused on the timeline.

Rogers, Bohlender, and Huff place the common ancestry of the Neandertal-Denisovan branch and the African ancestors of modern humans at 744,000 years ago. They acknowledge that this estimate depends on assumptions about the mutation rate, and those assumptions leave a lot of possibility for error—for example, they show that a faster mutation rate as preferred by some geneticists (5 × 10−10 per year) gives rise to a more recent estimate of only 616,000 years. That faster mutation rate is probably not correct, but whatever the mutation rate, it’s not appropriate to talk about 3 significant digits with estimates that have so many uncertain inputs. I prefer to say that the date is around 700,000 years.

The most striking aspect of the population model described by Rogers, Bohlender, and Huff is that Neandertals and Denisovans divided into separate populations quite rapidly after their common origin. This conclusion contrasts with earlier thinking, which suggested a slow divergence of Neandertals and Denisovans from each other after their common origin. Rogers and colleagues also find that the effective population size of this ancestral Neandertal-Denisovan population was very small. They propose one explanation is a population bottleneck and founder effect in the origin of these populations.

Another finding, which I have not depicted in the figure above, is that the Neandertal population had a quite large effective population size, maybe as large as 15,000 to 30,000 effective individuals. This also contrasts with previous estimates of Neandertal effective size.

I don’t consider this contrast to be very surprising because this new method is measuring something different from the earlier ones. Both this and earlier estimates are measures of the strength of genetic drift within Neandertals, but previous estimates from Neandertal genomes are measures of inbreeding within the ancestors of that genome, which is strongly affected by local population histories if the Neandertal population was subdivided. If introgression into Eurasians came from a different subpopulation of Neandertals than the Altai genome, Rogers and colleagues’ estimate of effective size will refer to the overall metapopulation of Neandertals, not the local history of the Altai genome, and will therefore be much larger. Neither of these estimates says much about the actual number of Neandertals that walked the earth, the census population size of Neandertals.

A critique based on singletons

How strong is any of this?

Today, PNAS issued a one-page comment by Fabrizio Mafessonia and Kay Prüfer on this work, together with a one-page reply by Rogers, Bohlender, and Huff. Mafessonia and Prüfer suggest that Rogers and colleagues overlooked one category of genetic variation in their analyses, which would affect the results.

As described above, Rogers, Bohlender, and Huff studied the patterns of variations that are shared among genomes, Neandertal, Denisovan, and modern. Mafessonia and Prüfer suggest it is necessary also to look at the genetic variations that are unique within one of the genomes and therefore are not shared among them. They provide a brief analysis that suggests looking at these “singleton” variations changes the results, making the data more consistent with a fairly long time of shared ancestry by Neandertals and Denisovans, and a smaller effective size for Neandertals than Rogers and colleagues had found.

In their reply Rogers, Bohlender, and Huff provide a new analysis that includes the singleton variants, from the exact same genomes used by Mafessonia and Prüfer. They show it is correct that using all the singletons makes a lot of difference to the outcome—although even in this case their results still suggest a much more ancient separation of Denisovans and Neandertals than has previously been found by other researchers.

But the singleton data generate some estimates that cannot be reconciled with the fossil samples. For example, the Denisovan genome includes many more singletons than the Neandertal genome. Looking at these singletons, it would appear that the Denisovan genome must come from an individual that lived much later in time than the Altai Neandertal—maybe within the last 4000 years. This is way off compared to the geological history of the site, which shows the Denisovan genome to be much older. There must be some other source of singletons in this genome other than the mutational history proposed in the population model.

One possible explanation is a factor that the model does not include: the introgression into the Denisovan genome by a “hyperarchaic” ghost population. Some portions of the Denisovan genome accumulated many more mutations than expected because they actually come from a population that diverged from ancestral Africans much earlier in time.

In their reply, Rogers, Bohlender, and Huff conclude that including the singletons is a bad idea because of such possible biases that are not included in the model.

However, this explanation points back to a problem with every method of examining these ancient genomes. A model that includes all the possible interactions between every population has more parameters than can be fitted to the data. Looking at singletons in these four genomes provides four more data points, and may open a view into models that have four more parameters. But to do substantially better, we will need many additional ancient high-coverage genomes, and we will need to look more closely at genetic variation among more modern human populations.

With all this considered, there are good reasons to hesitate before accepting the exact values proposed by Rogers and colleagues. The model is leaving out important aspects of population history.

The fossil record speaks

Still, there is another source of evidence about Neandertal origins, and it also suggests a much earlier timeline than previously thought.

Recent genetic and geochronological findings from the Sima de los Huesos sample show that early Neandertals were not what we once assumed. Meyer and colleagues (2016) showed that Sima de los Huesos nuclear genetic samples are unambiguously close to Neandertals, and not closely connected with Denisovans. This shows that the Neandertal and Denisovan populations must already have separated substantially earlier than the deposition of the Sima de los Huesos hominin remains.

The geochronology of Sima de los Huesos provides evidence that the fossils were deposited around 430,000 years ago (Arsuaga et al. 2014; Arnold et al. 2014). Meyer and colleagues (2016) looked at the chronology from the point of view of genetic data and came to a somewhat weaker conclusion:

Although it is difficult to determine the age of Middle Pleistocene sites with certainty, geological dating methods, as well as the length of the branches in trees relating the mtDNAs from femur XIII and an SH cave bear to other mtDNAs, suggest an age of around 400,000 years for the SH fossils. This age is compatible with the population split time of 381,000–473,000 years ago estimated for Neanderthals and Denisovans on the basis of their nuclear genome sequences and using the human mutation rate of 0.5 × 10−9 per base pair per year. This mutation rate also suggests that the population split between archaic and modern humans occurred between 550,000 and 765,000 years ago.

That is, the mtDNA timeline gives approximately the same result as the geochronology, placing the fossils’ age around 400,000 years, and Meyer and colleagues suggest this does not contradict the notion that Neandertals and Denisovans parted ways only between 381,000 and 473,000 years ago.

I disagree. The population split must in fact be substantially earlier than the fossils’ age to give rise to the pattern of shared alleles between the Sima de los Huesos sequences and the later Neandertals.

One way of looking at how early the Neandertal population arose would be to examine the number of derived alleles shared by the Sima de los Huesos genetic data and the other Neandertals, in comparison to modern humans and the Denisovan genome. This is not straightforward, though, because if the Neandertals or the Neandertal-Denisovan ancestral populations did experience bottlenecks and founder effects, the Neandertals will share a higher fraction of derived alleles as a result of suppressed incomplete lineage sorting, in addition to new mutations early in their evolutionary history. The Sima de los Huesos specimens show around 40 percent derived allele sharing with later Neandertals, in comparison to 70 percent or more derived allele sharing of later Neandertal specimens with each other. By contrast, the Sima de los Huesos specimens share only around 7-9 percent derived alleles with Denisovans—way less than they share with Neandertals, and reflecting a substantial shared history of drift between the Sima de los Huesos and later Neandertal samples. All this shared drift happened earlier than 400,000 years ago, but it’s not clear how much of that shared drift is sheer time, and how much may have occurred quickly during bottlenecks.

Genetic structure within the ancient Neandertals makes a difference. The later Neandertal population had strong regional differences separating the Altai and other genetic samples. The Denisovan population also seems to have had strong regional structure, reflected in the differences between the present-day introgressed sequences and the Denisovan genome we have from the fossil record. Did the earlier Neandertal population also have strong regional structure? If so, the Sima de los Huesos population itself may have been quite distinct from other contemporary Neandertals, and the shared ancestry of these regional populations might have preceded the Sima de los Huesos deposition by a hundred thousand years or more. If all the shared derived alleles of Sima de los Huesos and later Neandertals date to earlier than 500,000 years ago, their common evolution must have started even earlier.

Another element of the quote from Meyer and colleagues is their use of the higher 5 × 10−10 per year mutation rate, compared to 3.8 × 10−10 used by Rogers, Bohlender, and Huff. A 25 percent higher mutation rate obviously leads to a 25 percent lower estimate of genetic divergence.

All of this suggests that the separation time of Neandertals and Denisovans was indeed quite a bit older than most sources have suggested up to now. I do not believe that the estimate of separation time proposed by Mafessonia and Prüfer (2017), only around 460,000 years ago, can possibly be true. If Sima de los Huesos actually dates to around 430,000 years ago, as in the present geological chronology, the previous genetic estimates are simply too young.

The value for Neandertal-Denisovan separation time reported by Rogers, Bohlender, and Huff is one possibility, placing this divergence almost as old as the initial separation of the Neandertal-Denisovan ancestral population from ancestral Africans. That means Neandertals have existed as a population for more than 700,000 years. Or, as Rogers, Bohlender, and Huff find in their singleton analysis, the Neandertal-Denisovan separation time might be as recent as 630,000 years ago.

More recent than this seems doubtful in light of the shared genetic history of Sima de los Huesos and later Neandertals. But where to draw a line indicating a minimum possible date for the Neandertal-Denisovan separation is not clear.

What is clear is that the origin of Neandertal and Denisovan populations is much older than previously assumed. And that timeline makes a hash out of many long-standing ideas about the fossil and archaeological records. I’ll be writing about some of these ideas over the next few weeks, with some ideas about where the science must go next.


Arnold, L. J., Demuro, M., Parés, J. M., Arsuaga, J. L., Aranburu, A., de Castro, J. M. B., & Carbonell, E. (2014). Luminescence dating and palaeomagnetic age constraint on hominins from Sima de los Huesos, Atapuerca, Spain. Journal of human evolution, 67, 85-107.

Arsuaga, J. L., Martínez, I., Arnold, L. J., Aranburu, A., Gracia-Téllez, A., Sharp, W. D., ... & Poza-Rey, E. (2014). Neandertal roots: Cranial and chronological evidence from Sima de los Huesos. Science, 344(6190), 1358-1363.

Hawks, J. (2017). Neanderthals and Denisovans as biological invaders. Proceedings of the National Academy of Sciences, 201713163. doi:10.1073/pnas.1713163114

Meyer, M., Arsuaga, J. L., de Filippo, C., Nagel, S., Aximu-Petri, A., Nickel, B., ... & Viola, B. (2016). Nuclear DNA sequences from the Middle Pleistocene Sima de los Huesos hominins. Nature, 531(7595), 504-507.

Rogers, A. R., Bohlender, R. J., & Huff, C. D. (2017). Early history of Neanderthals and Denisovans. Proceedings of the National Academy of Sciences, 114(37), 9859-9863. doi:10.1073/pnas.1706426114