I had a great session with my advanced students yesterday running through different evolutionary scenarios for the X-Woman. This and some later posts will follow up on my initial thoughts ("Hobbit version 2.0: the undiscovered hominin").
May I just say, "X-Woman" is one of the more dopey nicknames for an ancient piece of bone? I mean, it's better than "Twiggy", but jeesh. I can't be the only one who thinks of John Singer Sargent:
Meanwhile, I have some great e-mails about Madame X, some of which I can share. First, an exchange on the topic of incomplete lineage sorting:
I'm confused by your suggestion of an ancient divergence among Neanderthals. Wouldn't that lead to a tree with the Siberian DNA and other Neanderthal DNA samples forming their own clade, to the exclusion of human DNA? As things stand, the Neanderthals are closer to humans than to the Siberian DNA.
Not at all; it could be either way.
Consider humans today. Africans have mtDNA lineages (the L clades) that are deeper in the human tree than any outside of Africa and basically absent elsewhere except for recent migration. But Africa also contains many of the mtDNA lineages that are present in Europe, India and West Asia.
Now imagine that the human population divides into two species, Africans and non-Africans, and those species persist for 100,000 years. Assuming no huge bottlenecks in either of these species, they both ought to retain the major clades present today. If we sample their genes at that time, 100,000 years in the future, we'll discover that Africans will be more genetically diverse than non-Africans. And the Africans will have L clades that are outgroups to the clades (M and others) that include some Africans and all non-Africans.
Subsequent population bottlenecks or selection could eliminate those ancient clades, but they will hang around unless they are eliminated. That's also the explanation for why humans and gorillas are more genetically similar at some loci than either is to chimpanzees, even though humans and chimpanzees speciated more recently. The variation in that ancestral H-C-G population was retained in the ancestral H-C population, to some extent, and lineage sorting sometimes gave humans the more gorilla-like clade.
An interesting question is whether the rest of the Neandertal sample would be so relatively invariant, if some part of their population included this quite divergent mtDNA haplotype.
It's quite hard to answer that question given the small sample of Neandertal mtDNA -- only less than twenty individuals, sampled from a range of times. A "lopsided" tree, with a lot of similar sequences and a few divergent ones, is not an unlikely genealogy in a small sample. The variance in the lengths of the deep branches in a genealogy is intrinsically high, even in the simple Wright-Fisher model with no population structure or selection. A "lopsided" tree is just one possibility on a continuum, in which the deepest coalescence time in the sample is high relative to the next deepest -- not an unlikely event at all.
For those who would like to explore this process, I put together a Mathematica demonstration ("Coalescent Gene Genealogies") that generates random gene trees under the neutral Wright-Fisher model. Strange-looking trees are normal, in the sense that they occur often enough that they are not statistically unlikely for a single gene locus.
Obviously what you'd want to do is compare multiple gene loci -- in this case, to get nuclear genomic sequence. Since the Max Planck group is actively pursuing further sequencing (and already has had some success, according to their press conference), I expect they're already making progress toward testing the neutral hypothesis.
If mtDNA proves to be unusual compared to other loci, then it's either intrinsic coalescent variability, or selection. Testing those two alternatives would require a larger sample of Neandertal mtDNA.
If, on the other hand, the nuclear genetic diversity is also substantially not shared with Neandertals (or living people), then the hypothesis of population structure in Late Pleistocene-age Eurasia would be strongly supported. It's a bit more complicated to test whether a speciation had occurred, but with whole genomes such a test can almost certainly be done.