New Denisova and Neandertal DNA results reported

Elizabeth Pennisi reports from the Biology of Genomes conference at Cold Spring Harbor, New York: "More Genomes From Denisova Cave Show Mixing of Early Human Groups". The article describes a talk by Svante Pääbo about new results from Neandertal DNA, as well as new analyses of the Denisovan genome. It has lots of details for those interested in these topics, but the article is paywalled, so I can only share a little of it here:

From the detailed genomes of both Neandertals and Denisovans, Pbo and Montgomery Slatkin of the University of California, Berkeley, estimated that 17% of the Denisovan DNA was from the local Neandertals. And the comparison revealed another surprise: Four percent of the Denisovan genome comes from yet another, more ancient, human"something unknown," Pbo reported. "Getting better coverage and more genomes, you can start to see the networks of interactions in a world long ago," says David Kingsley, an evolutionary biologist at Stanford University in Palo Alto, California.
With all the interbreeding, "it's more a network than a tree," points out Carles Lalueza-Fox, a paleogeneticist from the Institute of Evolutionary Biology in Barcelona, Spain. Pbo hesitates to call Denisovans a distinct species, and the picture is getting more complicated with each new genome.

We have been finding some of this in our comparisons of the genomes also. These were not isolated groups of ancient people, and some of them were more similar to living people than others. It is just wonderful to have more and more DNA coming out -- although that makes it hard to think we won't learn something new from high-coverage data that will require us to re-run various comparisons. That's the cost of discovery!

Meanwhile, the article sheds light on two interesting contradictions in the Denisova data. The analysis of the high-coverage data last fall Meyer:Denisova:2012 noted that the pinky bone genome is consistent with a very small long-term effective size, because of its limited genetic variation ("Denisova at high coverage". These results included a "drastic decline in size" around the time the Denisovans were estimated to have separated their population from the ancestors of living sub-Saharan Africans.

That result was curious in comparison with the mtDNA evidence. The Denisovan mtDNA is substantially more divergent from living human and Neandertal mtDNA, with an estimated time for the last common ancestor of mtDNA among these groups a bit more than a million years ago. In the initial analysis of the Denisova genome, Reich and colleagues Reich:Denisova:2010 pointed out that even a deep divergence might be consistent with a neutral population history in a single population. But a population of radically reduced size, with a substantially more recent common ancestry shared with Neandertals and other ancestors of living people? Seems odd.

Now, we may be learning that the Denisovan genome itself represents different ancestral groups -- not only a more ancient "something unknown" population, but substantially the local Neandertals. That kind of mixture is not the population history described by papers on the Denisova genome so far. And a third Denisovan mtDNA from one of the third molars at the site is substantially different from the other two, pointing to greater mtDNA diversity within the Denisovan population than now known from either Neandertals or living people.

What does it mean? I don't think there's a contradiction here in the data. What this shows is that the methods applied to the data have been too simplistic. The methods will come to a result, but that result may not fit the data as well as a population model with more complexity. Looking only at one kind of comparison -- as the Li and Durbin model applied to the Denisova genome by Meyer and colleagues last year Meyer:Denisova:2012 -- will probably not give a result that describes the true population history. We need to keep our minds open to more complex population histories that may be more consistent with other sources of data, including archaeological and fossil information.