Neandertals in NYC

20 minute read

I'm sitting at my gate at LaGuardia, returning from a conference at NYU, titled "Neanderthals revisited: New approaches and perspectives," cosponsored by the Max Planck Institute for Evolutionary Anthropology in Leipzig. The conference organizers were Katerina Harvati (now of Max Planck) and Terry Harrison (of NYU). Reflecting on the proceedings, I would say it was a marvelous experience. Everything was well-organized, from the scientific program and discussions to the accommodations and meals. I had some great evenings with good friends, and made many new friends as well. The conference reflects very well on its organizers and the NYU students who helped keep things running.

Conference program in PDF format

I'm going to take a while to digest things: I have about fifteen pages of notes focused on research questions that I will be pursuing in the next several months. I would say that the presentations focused on several themes. Most of the participants were relatively junior, which made for an interesting exchange of ideas. I missed some of my more senior friends, but it was a great chance to get to know many of my own cohort much better than I did before. It may be an illusion, but I would say my impression is that many of my younger colleagues have a keener interest in collaboration and interaction than most of us are used to seeing. I think this is a very positive thing for the science.

Homology and features

The conference was not a random selection of topics, and I see three important themes as having been strongly represented by the mix of papers and discussion. Many of the participants expressed doubts of one kind or another in our ability to accurately assess and compare features on skeletal remains. One source of problems is the fact that different researchers employ different criteria to determine whether a feature is present or not.

The suprainiac fossa is a notorious example. This fossa occurs as a depression on the surface of the occipital bone, immediately superior to the external occipital protuberance or inion on the rear of the skull. Most Neandertal skulls have such a depression, and it usually has a pitted surface as a result of bony resorption. Some more recent human skeletons have a depression here also. Most often, this depression in recent people does not exactly match the morphology in Neandertals--the fossa does not take on the form of an elongated oval, it does not show the same extent of resorption as the Neandertals, and it is more often associated with a projecting superior nuchal line, which does not occur often in Neandertals with an occipital bun.

So the question is, are these the same morphological character or not? The question is one of homology: are the traits in the two populations derived from a single trait in their ancestor (which in this case may include the Neandertals)? Judging homology by similarity alone, the suprainiac fossa in Neandertals can be differentiated easily from the depression found in many (although not all) recent humans. So there is a case to be made for considering these to be different traits altogether, even though both involve a depression in the same place on the same bone.

But another way to judge homology is by considering developmental or genetic aspects of the trait. There is no single gene that has different alleles coding for whether an individual will have a suprainiac fossa or not. The morphology is a result of a combination of different genetic, developmental, and ultimately structural factors. A skull that has a different topology of the cranial rear above inion will likely lack a suprainiac fossa, even in the absence of genetic differences. Consider the hypothesis that the depression in the skulls of some recent humans actually results from exactly the same developmental process displayed in Neandertals, but that the cranial rears of the two groups vary in topology, resulting in characteristically different formations of the character in the two populations. Under this hypothesis, the trait could be considered to be present in both populations despite the lack of similarity, and the occurrence of the trait could serve as evidence of population relationship.

Now, the hypothesis of similar developmental processes may not be true, but it does raise the question of how we should be defining our traits. In the end, defining a trait to study depends on assumptions--stated or unstated--about the genetic, developmental, and environmental factors affecting its expression. The most usual assumptions are that a trait is invariant once expressed in adults, that it is moderately heritable, that environmental variability in the trait affects its expression in mostly random ways (for example, in an unbiased way relative to the samples being compared), and that it is unlikely to be the result of more than a single genetic mechanism. These assumptions might or might not be true, and we probably need to be more cognizant of the iterated process of character analysis.

This observation is not merely relevant to the phylogenetic comparison of samples, but also to their functional interpretation. Consider the Neandertal nose. Bob Franciscus (University of Iowa) presented his research on Neandertal nasal breadth and facial projection. One of the main themes of his research is that a broad nose is not a consequence of cold adaptation in Neandertals. Students of the fossil record will know that the Neandertals themselves do not have the broadest noses in the hominids, and Franciscus has shown that nasal breadth does not show strong correlations with climate in recent people. On the other hand, the breadth of the upper nasal cavity does appear to have relevance to climatic adaptation--a trait reflected indirectly by the interorbital distance. The lower nasal breadth itself may be related in a complex way to facial projection and anterior tooth breadth and loading.

I can make a couple of observations about this work from the perspective of feature definition. As he set it up, Franciscus clearly logically began from the hypothesis that the two areas of the nose might have different functional roles in climatic adaptation. I don't know whether he took the measurements first or had the hypothesis first, but in a way the measurements and the hypothesis are inextricably linked. Without allowing the possibility that more than a single measure of nasal breadth might yield different information, there could be no hypothesis about the function of the upper and lower nasal cavity. I almost certainly wouldn't have thought of the hypothesis, and for that reason I would never have taken the measurements. Not many of us have looked inside the upper nasal aperture thinking about the function and measurement possibilities. Stated or unstated, hypotheses of process (such as targets of selection) lead our empirical research.

An ancillary point is the idea that a combination of different characteristics is more definitive than any single feature. Jean-Jacques Hublin raised this point in discussion, but it has been raised by many other students of the Neandertals. This is an important point in defining samples, because there is no single trait that can differentiate all Neandertal specimens from all non-Neandertal specimens in the fossil or comparative records. For example, even the most extreme Neandertal-like form of the suprainiac fossa is not found in all Neandertals, and it is found in at least one more recent European (Mladec 6). Most features are much more similar in frequency between Neandertals and recent people, especially Europeans. Thus, it takes several different features to accurately diagnose skeletal remains as Neandertal, and many (especially fragmentary) pieces that make sense as Neandertals in a spatiotemporal context nevertheless bear few or no features diagnosing them as such.

So often people say something like, "Sure, we all know you can find more recent humans that have an occipital bun, but you don't find any that have a combination of an occipital bun, suprainiac fossa, arching supraorbital torus, and midfacial prognathism. It is the combination of these features that are important, not a single one."

One response to this is the obvious: "Well, of course, if we had a skull with all those characteristics, we would call it a Neandertal whatever its date." In other words, the set of skulls that are interesting, from the perspective of being possible evidence of Neandertal phylogeny, is entirely composed of skulls that have a small number of Neandertal similarities. If they had more Neandertal similarities, then they would be Neandertals.

This is not a minor point. A skull like the Saint-Césaire cranium has several similarities with Neandertals. It also has several features that are more similar to recent humans than to most Neandertals, such as a more slight browridge, a narrower nasal aperture, and a chin. These features have been argued to be evidence for evolutionary change in the latest Neandertal populations toward a form more similar to recent humans. But suppose that the browridge of Saint-Césaire were even smaller, or significantly divided between central and lateral elements. Or suppose that the face projected slightly less anteriorly. How many Neandertal features would we have to erase to make the skull no longer a Neandertal?

When posed in that way, the question is a question of typology. But typology is irrelevant to the evolutionary process. Adding more and more features in combination with each other to make our comparisons is nothing more than statistical chest-beating. It is obvious that if the frequency of traits differ between two populations, adding more and more traits will make it more and more likely that two individuals in those populations will look different in our comparisons. As long as we exclude traits whose frequency is the same between the populations, there is no escaping that individuals in one population will look different from the other. This is the basis, after all, of discriminant function analysis--the idea that the combination of observations that are less likely in one population than another can lead to an accurate assignment of individuals to populations. Now, there are many possible scenarios in which it might take more or less traits to differentiate populations in this way. The point is that confirming a typology does not provide a statistical test of difference.

The test of a phylogenetic hypothesis is the level of gene flow. All other considerations are ultimately irrelevant. This includes:

  • how likely it would be to choose two equally divergent samples from a single population or species
  • how few characters it takes to divide the samples replicably into two groups
  • how long the differences were maintained between samples

Another possible observation, how adaptive or nonadaptive the differences might be, is potentially important. This is because the degree of selection on such traits might be an important consideration when we try to estimate the level of gene flow. For example, a long-standing differentiation between populations is unlikely in the presence of gene flow unless the difference is under selection.

This issue was one of the themes of my own presentation, focusing on the interaction of selection, gene flow, and population structure in Neandertal populations. I gave an illustration of the mathematical conditions under which a selected difference can be maintained in two populations connected by constant genetic exchanges. In my example, the low Neandertal crural index, the degree of difference between Neandertals and African populations--although vast compared to the within-population variability--could have been maintained by a very low level of selection. In my example, I was able to conclude that the long-standing adaptation to cold in Neandertal populations was no reason to think those populations were isolated from other global populations during the Late Pleistocene.


The conference put on display the ways that geometric morphometric methods have reorganized a substantial proportion of the research production in paleoanthropology. Scientists use morphometric methods to study shape differences among fossils and samples of skeletal materials. There are many different ways to study shape, ranging from principal components analysis and other multivariate methods applied to linear measurements to geometric analysis of landmarks and other three-dimensional studies. Some of these are very old, and some of them are more recent. There may be several factors underlying their current popularity; I count two of them as the increased emphasis on developmental processes as they affect shape changes in multiple anatomical systems, and the increased availability of digitizing tools and computer programs to process the resulting data.

A good example of the latter influence was the presentation by Emiliano Bruner (University of Rome, "La Sapienza"), who gave an examination of the endocast of Saccopastore 1, taken by CT. The centerpiece of the study was the comparison of details of surface endocast anatomy with earlier and later hominids. These kinds of comparisons depend on examination of shape differences in a three-dimensional schema, encompassing details such as petalial assymmetries, flattening or increased curvature at some points, and volumetric comparisons. Yet, although studies like this one rely on relatively recent geometric innovations, such as the use of "semi-landmarks" (arbitrarily chosen points on a shape that are mathematically smoothed into complex curves), their use is essentially descriptive. Morphometrics provide a way to quantify the degree of shape difference and localize those differences in multiple specimens with different possible alignments.

The use of distances in phylogenetic comparisons at different levels was illustrated by a paper from Katerina Harvati and Tim Weaver, both of the Max-Planck Institute. The question addressed by their study was whether morphometric distances were correlated with spatial or environmental characteristics of populations. They concluded that some components of the skull were strongly correlated with climate (using latitute or mean temperature as a proxy), while others appeared to be more strongly related to a matrix of genetic distances among populations. These they considered to be reflective of "population history." I use the scare quotes because it is not clear what factors might feed into this concept of population history. Clearly bifurcating descent of populations from a single common ancestor is one possibility, but I would tend to think that both craniometrics and genetics likely reflect a history of isolation by distance. It is also quite plausible that other selective factors not directly relevant to climate (such as level of technology, mobility, diet, and of course many others) are included in a broad concept of "population history." In any event, the work follows upon other morphometric (in the sense of quantitative genetic) comparisons by Roseman, Weaver, Harpending, and Relethford on whether the pattern of craniometric variation among populations can serve as a proxy for their molecular pattern of relationships. The answer thus far has been ill-resolved. It would appear that the level of craniometric differences among populations (broadly enough considered) do accord with their genetic relationships, but the likely existence of selected functional or structural units in the skull has not yet been fully resolved with this idea.

The oldest form of morphometrics is the examination of growth curves. The method dates at least back to D'Arcy Thompson, and Franz Weidenreich made use of the idea of ontogenetic trajectories fairly extensively in his notions of the evolution of modern humans from an archaic ancestor. With the increase in examination of CT data from fossil skulls, it has become possible to examine the three-dimensional changes in shape through ontogenetic pathways. This kind of analysis was presented by Christoph Zollikofer and Marcia Ponce de León of the University of Zürich. These studies used cross-sectional data from living hominoids (humans, chimpanzees, and bonobos) as well as data from fossil Neandertal specimens of a range of ages, comparing the ontogenetic trajectories that would account for the shape changes in the skulls as a consequence of age. An interesting finding is that these ontogenetic trajectories are essentially the same in recent humans and in Neandertals, so that most of the difference between these forms is the result of a small period of altered development very early in life, possibly before birth. This finding is not unexpected--for a long time it has been known that characteristic features of archaic humans like the browridge and lack of a chin were differences expressed not only among adults but in a less obvious form in children and even neonates as well. But the focus on the earliest period of ontogeny to explain the level of difference between Neandertals and humans places an emphasis on the combined effect of a very small number of developmental genes, rather than the complex interaction of many genetic and environmental factors.

Selection versus drift

This is the theme nearest to my heart, since I am most directly concerned with estimating the effects of evolutionary forces in past populations. For this reason, I may have read more out of small parts of many papers than others along these lines. Admitting that I may be overrepresenting this topic, here are some thoughts that came up.

Anyone who has followed Neandertal studies will recognize the fundamental division in the field between those who primarily are interested in Neandertal adaptations as a way of reconstructing their lives, and those who are more interested in Neandertal features as a way of testing their phylogenetic relationships. Scholars who are interested in why Neandertals came to look the way they do are generally interested in the pattern of adaptation that their skeletal anatomy reflects. More than for any other ancient hominid population, anthropologists have derived the behavioral and anatomical adaptations of Neandertals in detail. Sometimes they have been wrong, as research on the breadth of the nose and cold adaptation has revealed. But whether correct or incorrect, ultimately the goal of much adaptive thinking in Neandertals has been to reconstruct selective forces and pathways of change. For this reason, many who have been directly concerned with studying Neandertal skeletal adaptations have been agnostic about their phylogenetic relationships to later people. The idea is that the Neandertals can be studied in their own right as a case of a distinctive adaptive pattern without raising the issue of their phylogenetic relationship to other ancient or living people.

But a complication with studying Neandertals as a unique adaptive pattern is that not every aspect of their anatomy is easily explained in adaptive terms. Sure, Neandertal limb proportions are well-explained by cold adaptation. And the larger and more fully-worn incisors of Neandertals are well-explained by the use of the anterior teeth as tools, or as a "third hand," in a sense. But how are we to explain the horizontal-oval mandibular foramen, which occurs in 53 percent of Neandertal mandibles but only rarely in later people? And what about the suprainiac fossa, which is very distinctive to Neandertals and much rarer in other populations? These kinds of features show strong differences in frequency between Neandertals and other populations, but have no obvious functional or adaptive benefit to them. The link between these features and the European climate or the Neandertal behavioral pattern has not been established.

One hypothesis to account for such features in Neandertals is that they were not products of selection. Instead, they were established in the European population by genetic drift, accentuated by the genetic isolation of Neandertals and their European ancestors. The nonadaptive hypothesis, or the drift hypothesis, has been elaborated as the main hypothesis of Neandertal speciation from Middle Pleistocene humans. The idea of nonadaptive traits distinguishing Neandertals was raised in discussion by Jean-Jacques Hublin, and commented on favorably by his colleague at Leipzig, Mark Stoneking, who reflected that anatomists have begun to talk about drift as an influential force in the same vein that geneticists have done for some time.

The examination of adaptive evolution in Neandertals was well represented at the conference, by Bob Franciscus, Steven Churchill (Duke University), Osbjorn Pearson (University of New Mexico), and of course my own talk.

The thing I wanted most to point out was that the examination of adaptation in Neandertals was not a different kind of inquiry from their phylogeny. In fact, the two are intimately connected to each other. It is possible for researchers to investigate adaptation under the hypothesis that their phylogeny does not matter. In this case, the researcher chooses to hold the parameter of gene flow constant and instead to talk only about the effect of selection in their evolutionary model. But this kind of model breaks down when we begin to consider more than a single trait. It may break down in meaningful ways even when we merely incorporate the correlation between two traits, when the pattern of selection is different between them. As I tried to show, the consideration of a selected trait like crural index together with a selected trait like cranial capacity leads to very different conclusions when gene flow is not held constant, but instead is allowed to vary.

Likewise, examining traits under the assumption that they are not selected can lead to very misleading results. It takes a very small amount of selection to have the same effect as very large demographic changes, for example. Considering the mtDNA of Neandertals, it is clear that a very small degree of selection on the modern human mtDNA complement would easily have led to Neandertal mtDNA disappearance without any demographic change whatsoever. But in contrast, explaining the same fact in the absence of selection requires an extreme demographic change--the near-complete extinction of the Neandertals and their total isolation from their contemporaries outside of Europe (otherwise the Neandertals could have conveyed their mtDNA to those contemporaries and thereby to living people). Our conclusion about Neandertal phylogeny (that they were a distinct species that became extinct without issue) depends not on our observation (Neandertal mtDNA disappearance) but instead on a single assumption about which parameters will be included in our evolutionary model (no selection).

About this I will probably have more to say.

What wasn't represented

The conference, being small and intimate, could not include every possible permutation of the Neandertal problem, or of every research path relevant to Neandertal evolution. It was not an archaeology conference, and therefore problems related to Neandertal cognition and behavior were really only touched upon briefly by the participants.

To some extent, this is a shame, because it is impossible to talk about what happened to Neandertals without a detailed consideration of their behavioral repertoire. A paper like my recent one with Milford Wolpoff and colleagues, "Why not the Neandertals" would really not be interesting without the behavioral component. Excluding the question of Neandertal cultural abilities necessarily excludes them from the anthropological (as opposed to purely biological) realm, just as the assumption of no selection excludes many relevant hypotheses for their genetic evolution.

There was a brief opportunity for two important Paleolithic archaeologists, Randall White and Alison Brooks, to address the conference during discussion. I think the thing to take away from their comments is the complexity of the problem of archaeological interpretation, especially given the limitations of the evidence. I would love to be able to see a detailed discussion between Francesco d'Errico and either of these two about the merits of their respective arguments for Neandertal cognition. Hopefully such opportunities will be afforded in the future.

Interestingly, there was relatively little discussion of the nature of the early Upper Paleolithic Europeans. There are two interesting points to make here. One is that the anatomical resemblances, such as they exist, between the Neandertals and the population that immediately succeeded them in Europe are obviously the most relevant test of both their phylogenetic relationships and the changes in the pattern of selection that occurred. I made this point as I talked about the evolution of cold adaptation and the evidence that the anatomy of Upper Paleolithic Europeans was not adequately explained by replacement, but must involve a selective explanation. Jim Ahern (University of Wyoming) also presented research along these lines, focusing on whether the frequencies of features between Neandertals and later humans are likely to differ as greatly between human groups in historical cases of partial population replacement, focusing on the New World.

But many essential faces were missing that could have addressed this issue. By far the most important of these is David Frayer, whose research on the changes in trait frequency and in metrics across the late Neandertals through the Upper Paleolithic sets the standard. Other people who might have talked about such topics include Erik Trinkaus and Fred Smith, who could not come at the last minute.

Final thoughts

In all, I thought that the conference was a great success, and I am really looking forward to my next opportunity to interact with this group. I put some brief words in another comment that are worth looking at as well.

Up to Neanderthals Revisited

Other meeting news