I just read the new paper by Philipp Gunz and colleagues, titled, "Early modern human diversity suggests subdivided population structure and a complex out-of-Africa scenario". That's a mouthful.
The late Middle Pleistocene population of Africa was genetically variable, and that genetic variability is probably the biggest component of genetic variation still remaining in living humans. Moreover, the phenotypic variability of the Levantine sample has been recognized since its initial description by McCown and Keith (1939). So to read this is not surprising:
Seemingly ancient contributions to the modern human gene pool (36) have been explained by admixture with archaic forms of Homo, e.g., Neanderthals. Although we cannot rule out such admixture (37), the clear morphological distinction between AMH and archaic forms of Homo in the light of the proposed ancestral population structure of early AMH to us suggests another underestimated possibility: the genetic exchange between subdivided populations of early AMH as a potential source for ancient contributions to the modern human gene pool (9, 36).
I've stressed the importance of African population structure before (e.g., Hawks et al. 2008). So I agree completely with this part of the interpretation in the paper: African variation was larger than in other regions, and it was important.
But that being said, these morphometric comparisons are not very straightforward. Some comments:
Phenotypic variance is not a measure of genetic variance. If we see a population that has a large measure of phenotypic variability, it does not mean that the population had much genetic variability. Perversely, genetic variability can sometimes be lower in a population that has greater phenotypic variance -- often because genetic drift can cause a loss of epistases that once constrained the phenotype. In some cases environmental variance may actually increase when the additive genetic variance declines, because of a loss of developmental robusticity. In any event, we can't just go from a variable phenotype and infer that there's variation in genotypes.
There's no evidence for subdivision here. They measure a high phenotypic variance within the sample they refer to early modern humans. But that variance is expressed not mainly between geographic locations in the sample, but within them. Qafzeh 6 and 9 are far apart; Jebel Irhoud 2 and Skhul 5 are close together. The East African fossils Omo 2 and LH 18 are far apart. This isn't subdivision, it's just high within-population variance.
Weird sample composition. The early modern human sample includes the African and Levantine crania complete enough for analysis. But why lump these? Why is the South African Fish Hoek skull lumped with Upper Paleolithic Europeans?
Temporal range. There are two samples here that have a high average distance between nearest neighbors in the sample: "archaic" humans and early modern ones. What these two samples have in common is that they each cover a much larger range of time than the other samples. The early modern sample spans more than 100,000 years by current dates. That's more 80,000 years longer than the Upper Paleolithic sample, 50,000 years longer than the Neandertal sample -- a huge component of variance that is uncontrolled in the other samples.
Principal components. PC axes are those that account for the largest covariances in the sample. If two samples are lumped together, there is a within-population component of variance and a between-population component. These may be partly independent in their effects on the total variance, or they may not be. In any event, if we derive the PC structure from the total sample, or even from the individual samples pooled together, the larger samples will weight the PC structure more toward the factors that explain their within-sample covariances. In this case, we have many more recent humans than fossil ones, and many more archaic humans and Neandertals than "early modern" humans. It's hard to have an intuitive idea about the biases that can result from sample composition, and that's a big reason for caution.
Those are all reasons for re-examining the results in different ways. In particular, if I were doing this kind of analysis, I would repeat it for subsets of the cranium, where I could include a larger number of fragmentary fossils. If the African-Levantine sample is really unusually variable, that should hold up strongly when we examine parts as well as the whole cranium.
Well, although I listed several reasons for caution, we can ask how to interpret the study's conclusion:
Any model consistent with our data requires a more dynamic scenario and a more complex population structure than the one implied by the classic Out-of-Africa model.
If we take the high variance of their "early modern" sample at face value, what we have to conclude is that later humans evolved substantially less phenotypic variance than African-West Asian people who lived between 200,000 and 90,000 years ago. Genetics tells us that there was no massive genetic drift during the time span after 90,000 years ago within Africa. Thus we must conclude that some other force resulted in a significant restriction of the phenotypic variation of recent humans, including people who lived as long as 40,000 years ago.
My hypothesis would be natural selection on some significant subset of phenotypic characters, which reduced the phenotypic variance of most of the cranium by pleiotropy. An out-of-Africa migration is not sufficient to explain the reduction in variance, because all modern humans are limited in phenotypic variance, not only non-Africans. Selection on some significant set of genes would help to explain why the ancestral African population predominated within the last 100,000 years. This selection would have predated most of the recent acceleration we observe in the genomic variation of current populations -- indeed, whatever set of genes was strongly selected before 50,000 years ago might have been fixed long ago.
A wave of selection can promote dispersal and demographic growth without the necessity of complete population replacement (cf. Eswaran 2002). A substantial transition in the genetic background would alter the phenotypic effects of any genes that remained in non-Africans from their local ancestors. In other words, the answer about what happened to fossil humans outside of Africa depends on the kind of events that happened inside Africa. So from that perspective, this research is very interesting.
Gunz P, Bookstein FL, Mitteroecker P, Stadlmayr A, Seidler H, Weber GW. 2009. Early modern human diversity suggests subdivided population structure and a complex out-of-Africa scenario. Proc Nat Acad Sci USA (early online) doi:10.1073/pnas.0909160106
Eswaran V. 2002. A diffusion wave out of Africa: the mechanism of the modern human revolution? Curr Anthropol 43:749-774.
McCown TD, Keith A. 1939. The Stone Age Man of Mount Carmel: The fossil human remains from the Levalloiso-Mousterian. Clarendon Press, Oxford.