How much sex did it take for Neandertal DNA to enter modern populations?

I’ve given a lot of thought over the years to the genetic ties between today’s people and ancient populations. Just last month, I wrote a lot about the relationships of Neandertal and modern human populations (“Earlier mixture from modern humans into Neandertal populations”). Last week, three new papers came out that bear heavily on this topic, and I’ve been writing quite a lot more.

I’ve seen a widespread misconception about Neandertal sex. I follow many professional colleagues on Facebook, some of whom are highly respected for their work but maybe not so up-to-date on genetic methods. I have seen a few of them taking up this misconception and spreading it through their professional networks. I won’t point to those social media posts here, because their authors did not intend them for widespread distribution, but I do want to address the misconception.

Their idea is that geneticists are trying to count the number of sexual encounters between modern humans and Neandertals. Not the number of contacts between populations, but an actual coital count. Like they are reading the credits of a porno movie.

Where did they get such a kooky idea?

Unfortunately, when writers have discussed Neandertal and Denisovan introgression, it is common to see them refer to a “distinct number of matings” or some similar phrasing. For example, in Science last week, the excellent Ann Gibbons wrote:

By developing powerful new statistical methods, an international team has identified how often and on which continents modern humans, Neandertals, and a second kind of archaic human called Denisovans met and mated. The researchers conclude that if you're an East Asian, you have three Neandertals in your family tree; Europeans and South Asians have two, and Melanesians only one.

“Three Neandertals in your family tree” makes it sound as if only three Neandertal individuals were part of the ancestry of any human populations today. The message is unfortunately reinforced by the headline (and I know writers do not usually choose their headlines):

Five matings for moderns, Neandertals

Again, that emphasizes the idea that somebody is counting marriages, or couplings, or some discrete number of coital episodes.

This is a misconception. Nobody is counting the “matings” between Neandertals and modern human populations.

Gibbons elsewhere refers to a “rich sexual history” of humans and Neandertals, which is a phrase I like quite a lot. But some anthropologists, confused by the misconception about counting the matings, noted that five matings is hardly a rich history!

In fact, we do not know how many sex acts are represented by the Neandertal ancestry within human genomes. The question itself reflects a kind of biblical, pre-evolutionary thinking. The underlying assumption is that races spring from individuals, like Ham, Shem and Japheth, and so the sex acts of a few individuals can explain the phylogenetic patterns of later populations. This is false, three episodes of Neandertal coitus cannot account for the present distribution of Neandertal genes in living populations.

We have a fairly clear idea of what fraction of the ancestry of today’s people came from Neandertals; this fraction was around about 2% for most people worldwide, a bit more for many of the populations of East Asia, and very little for most populations of sub-Saharan Africa today. Vernot and colleagues’ new paper (Science 10.1126/science.aad9416) helps to confirm that the elevation of Neandertal genetic similarity among East Asian populations is the result of additional mixture that does not characterize the ancestry of today’s western Eurasian populations. The authors further infer that all of today’s Eurasian samples share evidence of Neandertal mixture above and beyond that present in today’s Melanesian populations. By showing that there are at least three different levels of Neandertal ancestry in different populations, with partially non-overlapping haplotype distributions, they have demonstrated that admixture must have occurred at a minimum of three places and times, among ancestors of three different subsets of today’s people.

How many hybrids?

I’m going to do some napkin calculations about the number of individuals necessary to deliver our Neandertal ancestry. How many hybrids do we need? The exercise is unrealistic because we have to make some assumptions that I find to be unjustifiable. But it’s easy to show that the answer is a lot more than three, and probably is more than a few thousand.

Who were hybrids? First, let’s assume that Neandertal genes came into the modern population only through the vehicle of Neandertal-modern F1 hybrids who were uniformly 50% Neandertal. This corresponds to the minimum possible amount of interbreeding between the two varieties, and the assumption rules out staged admixture, substantial modern gene flow into Neandertals, and long-term gene flow between African and Neandertal populations as they originated. I’m unwilling to rule out these alternative scenarios. All of these alternatives would tend to increase the amount of interbreeding necessary to account for today’s ancestry fraction from Neandertals. I also recognize that many people wouldn’t use the term “hybrid” at all for these individuals.

Effective population size. We further have to assume something about the proportion of effective population size to census population size. The effective size of a population is a measure of the rate of genetic drift it experiences; this rate is determined in part by nonrandom mating within the population and in part by its size. The contribution of nonrandom mating means that the effective size is usually different from the number of breeding individuals, it may be quite a lot smaller. I have a great book chapter explaining this concept in more detail on Academia.edu, if you are interested in how we relate genetics to population sizes.

The concept of effective population size allows geneticists to use the “effective” number to talk about drift and loss of variation without needing to know the exact ratio of genetic variation to the true, or census, population size. But in the case of hybridization and introgression, drift may often have a different effect on the hybrids than it does on the population at large. This makes it difficult to estimate the ratio of hybrid to non-hybrid individuals even when we know the genetic effect of the hybrids. For simplicity, I’ll again go with an unrealistic assumption: that the ratio of effective to census numbers is the same for the hybrids as it is for the modern population as a whole.

Initial admixture. Let’s consider first the population that gave rise to all modern humans outside Africa today. If the mixing with Neandertals had been instantaneous, in a single generation, that generation would have included approximately 4% F1 Neandertal hybrids.

The effective size of the ancestral non-African modern population at the moment of such mixture was small relative to today’s populations. Indeed, our best models suggest that the effective size of this population may have been as small as 10,000 effective individuals at one point. This is what is usually termed the “genetic bottleneck” leading to the settlement of areas outside Africa. We do not know that this genetic bottleneck was really a single reduction in numbers, but clearly this ancestral population was separated from sub-Saharan African populations for some time and underwent substantial genetic drift, which may even reflect multiple events over time.

Importantly, there is no sign that the population was ever much smaller than 10,000 effective individuals, and that puts a lower limit on the number of hybrids that must have been introduced into this population to account for the Neandertal ancestry of its descendants. If these ancestors mixed with Neandertals during the minimum of such a bottleneck, then the effective number of F1 hybrids responsible for this mixture may have been as small as 400. But if mixture happened at some other time than the minimum, or if mixture happened over some longer period of time, then the effective number of F1 hybrids must have been larger, possibly more than an order of magnitude larger.

Later admixture. The same considerations apply to the ancestral populations that led to western Eurasian (south Asian and European) and East Asian population samples, each of which experienced additional admixture from Neandertals. The additional Neandertal ancestry present in today’s East Asian populations would have required another 1-2% F1 hybrid representation in their ancestral population.

Interestingly, the number of F1 hybrids underlying the small bump in Neandertal ancestry in East Asians might actually be greater than the number underlying the initial mixture within the common ancestral population of all non-African populations. This is because the effective number of F1 hybrids necessary for this later admixture again depends on the effective sizes of these populations. The genetic data suggest that these ancestral populations experienced a lower level of drift when compared to their common out-of-Africa ancestor. This may mean a larger effective population size for these East Asian and other populations, a population in which 1-2% F1 Neandertal hybrids may have actually been many hundreds or thousands of individuals, not the 100-200 that would correspond to a very small effective size.

But none of these effective numbers are actually known, and I have little confidence that we can estimate them from today’s simplistic population models. All of them almost certainly grew over time, so that the timing of mixture makes a large difference to how we estimate the number of hybrids that contributed to it. Early hybrids probably made a much larger relative difference than later ones.

We cannot talk about effective numbers of F1 hybrids without recognizing that the effective sizes of human populations are substantially smaller than their census sizes. If the relation is the same for Neandertal-modern hybrids, then we may be looking at several true individuals for every “effective” individual. For a total effective number of 600-1000 F1 hybrid individuals, which is a bare minimum, this might mean upward of 2000-3000 actual F1 hybrids. But then all of my assumptions to this point have been unrealistic, all minimizing the extent of interbreeding between populations. In reality, many more individuals must have been mating, over a much longer span of time than a single generation.

Some of these hybrids were the products of Neandertal love affairs. Many were the daughters and sons of Neandertal wives or husbands who spent long passionate lives with modern mates. Some were likely the children of captured Neandertal slaves. Some were siblings, so the number of Neandertal mothers or fathers was to some extent smaller than the number of hybrids introduced into modern populations.

So if you ask me how many hybrid individuals may have been direct ancestors of today’s populations, I think the number is minimally close to a thousand and likely many thousands. And if you ask me how many Neandertal sex acts took place, I suppose I’ll smile and ask, “Who wants to know?”

References

Gibbons A. 2016. Five matings for moderns, Neandertals. Science 351:1250-1251. doi:10.1126/science.351.6279.1250

Hawks J. 2008. From genes to numbers: Effective population sizes in human evolution. Pp. 9--30 in Bocquet-Appel J. ed., Recent Advances in Paleodemography Springer, New York. Academia.edu reprint

Vernot B et al. 2016. Excavating Neandertal and Denisovan DNA from the genomes of Melanesian individuals. Science (online) doi:10.1126/science.aad9416