Secrets within the teeth of the first Homo fossils

I have to be honest: When I started out in human evolution research, studying teeth was not my dream. The bones of the skull and the postcranial skeleton impressed me with what they could say about ancient lifestyles and relationships. Squeezing information out of teeth would take special techniques that were just starting to be developed. Lucky for everyone, I have had amazing colleagues over the years who have found stories in ancient teeth that would have stayed hidden without them.

A new study published last week reveals some key information from the teeth of Homo habilis and other early Homo fossils. The study, led by Thomas Davies, shows that most of these Homo teeth have a shape and structure much more like Australopithecus than like Homo erectus or other later forms of Homo.

This new research adds to a similar recent study of South African teeth, together highlighting old questions about the relationships of early Homo. We do not know which species was the ancestor of our genus. The earliest fossils attributed to Homo are almost 800,000 years older than any that are complete enough to enable diagnosing a species. What ties many of the early fossils together is their teeth. But as these new studies show, the teeth may not be as helpful as many scientists have assumed.

A history of early Homo

Homo habilis has been challenging scientists since Louis Leakey, Phillip Tobias, and John Napier named it in 1964. As they came to understand the hominins from Olduvai Gorge, Tanzania, they felt some fossils did not fit within either Australopithecus africanus or Homo erectus, the two closest species then known. The brain sizes estimated for OH 7, OH 13, and OH 16 were between those of Australopithecus and H. erectus. The molar and premolar teeth also were between the averages of Au. africanus and H. erectus, overlapping with the Australopithecus range. The skulls lacked the elongated shape of H. erectus skulls. Ranking the brain size as the most important observation, Leakey, Tobias, and Napier didn't only name a new species; they wrote a new definition of the genus Homo tailored to match the features of the new fossil finds.

Some anthropologists welcomed H. habilis as an intermediate between Australopithecus and Homo erectus—one more link in a line of progress toward modern humans. That idea, which was more or less Tobias' view, became the most common textbook version for the next fifty years. But Louis Leakey himself disagreed. He thought it likely that H. habilis was one branch, possibly related to living humans, and H. erectus another separate branch. Many others saw the new species as nothing special. John Robinson—at the time, responsible for describing more Australopithecus fossils than anyone else—was the most vocal of many researchers who maintained that the new Olduvai fossils fit within Au. africanus.

“I submit that morphologically it is almost impossible to regard H. habilis as representing ‘a stage between Australopithecus africanus and Homo erectus.’”—Louis Leakey

Richard Leakey added more evidence when his team began to discover new fossils from the Turkana Basin. Some of the Turkana fossils had brain sizes like the Olduvai Homo habilis skulls, and shared the rounded shape different from H. erectus. Most experts quickly accepted them as new members of this species. Meanwhile in South Africa, Alun Hughes unearthed a fragmented skull from Sterkfontein that seemed to share the same rounded shape. Tobias saw this StW 53 skull as a southern member of H. habilis.

Yet throughout the 1970s and 1980s, there were researchers who raised questions about whether these fossils might really be different things. For some, the small KNM-ER 1813 skull belonged to Australopithecus; others thought it was a tiny version of Homo erectus. One opinion with staying power was that the large KNM-ER 1470 skull is different from the others. The Russian anthropologist Valery Alexeyev named it Pithecanthropus rudolfensis, and the species name stuck, now usually Homo rudolfensis. By the early 2000s, Meave Leakey had led the discovery of a handful of other Turkana Basin fossils that seemed like matches to the flat facial shape and large tooth sizes of the 1470 skull, lending some weight to the H. rudolfensis idea.

Over time, some anthropologists began to think that Leakey, Tobias, and Napier made a mistake in expanding our genus Homo to include the habilis fossils. The body size, brain size, and teeth of Homo erectus seemed much more like humans, and what was known about these features in Homo habilis and Homo rudolfensis much more like Australopithecus. A few began to use the names Australopithecus habilis and Australopithecus rudolfensis.

That trend didn't catch on. New discoveries of Homo species with small brain sizes much later in the Pleistocene showed that big brains are not the defining feature of our genus.

Today, the place of Homo habilis and Homo rudolfensis on the hominin phylogeny is uncertain. Some analyses connect Homo habilis with the South African Australopithecus sediba. Others link Homo rudolfensis with the much earlier Kenyanthropus platyops. It's hard to be very sure of anything without much more evidence from across the skeleton.

But scientists' knowledge of the postcranial anatomy of H. habilis is based on a tiny number of associations: the hand of OH 7, a partial skeleton numbered OH 62, and a few bones of KNM-ER 3735. All suggest a small body size and stature, more or less like the “Lucy” skeleton of Australopithecus afarensis. But the attribution of OH 62 and KNM-ER 3735 to H. habilis is disputed by some researchers. There's just little to go on. That leaves the teeth as the most promising avenue of analysis.

Looking inside the teeth

Fossil teeth make up large part of what we know about ancient human relatives. The enamel and dentin (also spelled dentine) that make up teeth are the toughest materials in the body, and they sometimes survive even in situations that can destroy bones. Teeth are also very recognizably different between different kinds of mammals. For their size, they are packed with information about diet, development, and phylogenetic relationships. In a collection of fossil fragments, paleontologists don't tend to miss hominin teeth, while they are less likely to recognize equally small chunks of most other bones. All this means that if you're going to study fossil hominins, you've got to love teeth.

But information erodes from teeth over the course of a lifetime. Food and grit wear down the occlusal surfaces over time, causing their shape to flatten. Eventually the wear penetrates the enamel and exposes the dentin deeper inside the tooth. This wear obscures the slight shape differences between species of hominins. The front (mesial) and back (distal) edges of teeth also wear against the neighboring teeth, actually making each tooth shorter over the course of a lifetime.

MicroCT scanning has opened methods of shape comparison of the internal structure of teeth. The most evident internal structure is the boundary between the enamel and dentin, called the enamel-dentin junction, or EDJ. Because it is beneath the hard enamel cap, the EDJ persists in its original shape until the enamel is worn almost completely away. The shape of the EDJ is related to the shape of the outside of the tooth, but has its own characteristics. Researchers around the world who study EDJ morphology have become experts at comparing the shapes of these hidden structures.

Most recent studies of EDJ morphology have used a geometric morphometric approach. Researchers identify consistent anatomical landmarks on each tooth, such as the tips of the dentin horns, and also use software to define strings of landmarks along curving structures. With the three-dimensional coordinates of each landmark, the researchers apply an algorithm that rotates and resizes data from every individual to minimize the squared distance each individual and the mean landmark configuration. The positions of landmarks will tend to covary: for longer teeth, all the distal landmarks are further distal, for example. The covariance can be described by principal components (PCs). Teeth from different individuals with similar form will map close to each other in PC space.

This is a different way of looking at teeth than the more traditional measures of crown length and breadth. It sometimes happens that teeth of very different sizes have similar EDJ shapes. Both shape and size are products of evolution, and while they are correlated they may provide information about different aspects of a species' history. Both also vary within species.

EDJ morphology of Homo habilis

The new research from Thomas Davies and collaborators adds some valuable information about H. habilis and where it may fit in the hominin tree. The study focuses most closely on the teeth from Olduvai Gorge, which were the first sample to be discovered. It includes some of the fossils from the Turkana Basin, including two with larger tooth sizes that sometimes have been attributed to Homo rudolfensis.

The study has some important limitations. It not include some of the more recent discoveries like KNM-ER 60000, KNM-ER 62000, or OH 65. These each represent larger individuals than most Homo habilis and would be valuable in examining the nature of H. rudolfensis. The most complete skull attributed to H. rudolfensis, the KNM-ER 1470 skull, lacks tooth crowns. Most heartbreaking is OH 62, the partial skeleton from Olduvai Gorge that many researchers attribute to H. habilis. Its tooth crowns are so cracked and worn that they provide no EDJ comparison.

Davies and colleagues find that most H. habilis teeth fit within or near the variation of Australopithecus. That makes the H. habilis situation different from H. erectus, which shows a different pattern of EDJ variation, including some derived aspects shared with Homo sapiens. The teeth belonging to H. habilis mostly don't share much with other species of Homo in the study. That's a result that would not have surprised John Robinson, who always maintained that H. habilis was not different from Au. africanus. Nor, I think, would it have much surprised Phillip Tobias, who pointed out some differences in the teeth of H. habilis but based most ideas about the species on the rest of the skull.

“The lack of clearly derived H. habilis dental traits at the EDJ in most tooth positions, combined with the overall primitive postcanine EDJ morphology in a number of key H. habilis specimens, suggest an overall Australopithecus-like endostructural dental morphology in the earliest members of the genus Homo.”—Thomas Davies and coworkers

The study's discussion raises a longstanding question: If H. habilis isn't different from Australopithecus, should the species be reassigned from Homo into that genus? The authors cite some of the turn-of-the-century research that suggested the Australopithecus habilis designation. They conclude that the placement of H. habilis should rely on a broader array of observations than the EDJ. From my own point of view, finding EDJ morphology like Au. africanus or Au. afarensis shows us that tooth evolution was conservative in many hominins—meaning that if we try to rely too much on teeth, we're likely to miss important parts of the pattern.

The study shows that the fossils attributed to H. habilis have exceptionally variable EDJ form, more variable than living species like humans, chimpanzees, or gorillas. Time seems to matter, since early and later fossils are more different than fossils of the same age. Among the later part of the sample, in particular the OH 16 fossil seems to stand out. This individual is represented by most of the upper and lower teeth, as well as a fragmented cranium, and is from the lower part of Bed II of Olduvai Gorge around 1.7 to 1.8 million years old. Davies and collaborators suggest it may possibly be something different.

What strikes me about the study is how conservative the EDJ of Australopithecus seems to have been. The results show fossils of Au. africanus and Au. afarensis interspersed with each other, the two species differing little despite the variation in tooth sizes. The variation within “recent Homo”, and in particular the variation within Neandertals, is quite a bit more than Australopithecus for most tooth positions. It's counter-intuitive, and makes me wonder what a similarly-scaled comparison of EDJ variation in great apes would look like. That is, are recent humans the outliers, or is Australopithecus?

Earliest Homo

Davies and coworkers were able to include one of the earliest fossils attributed to Homo, a partial maxilla from Hadar, Ethiopia numbered AL 666-1, estimated to be 2.33 million years old. The fossil was described by William Kimbel, Donald Johanson, and Yoel Rak in 1997. That description emphasized the traits that align the maxilla with Homo habilis, including the parabolic shape of its tooth row, the short length of its palate, and the more inclined profile just beneath the nose. Kimbel and collaborators stopped short of claiming that the fossil is the first Homo habilis, but that is where they saw the closest comparison.

Davies and coworkers found that the teeth of this individual were not much like Homo at all. When they used canonical variates analysis to compare Australopithecus, Homo erectus, and Homo habilis fossils, the AL 666-1 teeth were consistently placed with Australopithecus.

I'm not too surprised by those results. The sizes of the AL 666-1 teeth are not particularly like H. habilis; its molars approximate the largest H. habilis teeth and the premolars are larger than any. In size, they fit with Australopithecus africanus. The maxilla shares some shape characteristics with Homo, and what those aspects signify is a shortened snout. Fossils of Homo do share those characteristics, but so do some fossils attributed to Paranthropus. That would not necessarily mean that AL 666-1 is Paranthropus; to me, what it means is that these traits are not especially linked to a reduction in tooth size. I think they reflect a change in incisor and canine function that is shared both by Homo and Paranthropus.

That's why I'm not surprised that the AL 666-1 premolars and molars may look like Australopithecus. What would be really interesting is to compare this fossil with Australopithecus sediba.

South African early Homo

Two years ago, many of the same group of authors collaborated on a study of the EDJ morphology of teeth from the Early Pleistocene of South Africa. That study, led by Clément Zanolli, had a provocative conclusion: Scientists may have been overdiagnosing Homo in South African cave sites.

“Following our revision, among the 23 specimens previously labeled as Homo, only between 4 and 7 specimens very likely belong to this taxon.”—Clément Zanolli and coworkers

In some ways, the results of the study by Zanolli and coworkers made a lot of sense. I have spent a lot of time working to understand the variation of fossils from the South African sites. Many of those sites have fossils with geological ages estimated between 2.5 million and 1.4 million years ago. At Sterkfontein, Swartkrans, Drimolen, and Kromdraai, anthropologists have identified some fossils as Homo. But few of those preserve the same elements to enable a comparison. Few, that is, except for teeth.

Many teeth are all by themselves in a jumble of breccia. Other individuals in the caves are represented by two or three teeth in a broken maxilla or mandible. Only a few are relatively complete skulls or jaws. Initially, Broom and Robinson recognized that some Swartkrans individuals were smaller-toothed than Paranthropus, and considered them connected with Homo. One jaw they made the holotype of a species they called Telanthropus capensis; later Robinson accepted that this was likely Homo erectus and most recent scientists have agreed.

Surprising to me, Zanolli and coworkers found that SK 15 actually has EDJ morphology that looks like Paranthropus. Another squashed skull that Broom and Robinson had considered as Paranthropus, SK 27, is actually Homo-like, an attribution first emphasized by Ronald Clarke in 1977. The StW 151 individual, which died as a child and is represented by most of its teeth, looks like Australopithecus. So does StW 53, the fossil skull that Tobias was convinced belonged to Homo habilis.

At Drimolen, Zanolli and collaborators did not include a large fraction of the teeth attributed to Homo. But out of the four fossils from Drimolen that they looked at, the study found that every one should be reclassified as Paranthropus. It would be valuable to see whether the full sample shares this same morphology. With recent work showing that P. robustus at Drimolen may have some differences in form from those from those from Swartkrans and Kromdraai, it is possible that tooth size and some aspects of shape are also out of the usual P. robustus range.

When this work came out almost two years ago, it made me do some hard thinking. Many researchers over the years have noted that tooth size is not a very good way to divide the South African samples. Yet tooth size may be all that can be measured externally for worn teeth. Considering the internal data just confirms that the different species overlap in size a lot, including overlap with Homo. It seems very credible that many fossils have been misdiagnosed. Still, the results suggest that three-fourths of the Homo fossils are something else.

The new work by Davies and collaborators softens this hard conclusion to some extent. East African Homo habilis has EDJ morphology like Australopithecus. That includes the type specimen, OH 7. So a South African Homo habilis might well look like Australopithecus in its EDJ morphology also. Looking like Australopithecus in the EDJ is just evolutionary conservativism for early Homo.

But looking like Paranthropus may be another matter. Unfortunately these two studies do not overlap entirely in their samples, so it's hard to see whether any of the East African Homo fossils may be more similar to Paranthropus instead. What would be helpful is a study that included all the fossils: Homo, Australopithecus, and Paranthropus.

A look at the detail of these two studies shows some problematic differences. For example, the studies come to completely opposite conclusions about the SK 27 skull. Zanolli and coworkers concluded that the SK 27 fossil EDJ is close to their later Homo sample, but most of the comparisons from Davies and coworkers place the fossil near Australopithecus. Both the results and comparative samples are different bewteen the studies, and those differences must explain the different outcomes. But it's hard to say which outcome is accurate, or whether a fuller sample might come to yet a different result.

Challenges ahead

Fossil teeth can provide a bigger sample than any other part of the body. That's a great thing! Within-species variation is the key to understanding the direction and speed of evolution, and the more fossils we have of each species, the more we can understand their variation.

But the more fossils are included in a study, the more it matters to consider possible biases leading some fossils to be omitted. Many important fossils are missing from these studies. The Dmanisi Homo erectus fossils, some of the earliest anywhere, are not included. Nor are several early H. erectus fossils from Ethiopia and Eritrea, or the key fossils attributed to Homo rudolfensis from Kenya. Knowing the authors of these studies, I know they would love to have every one of these in their samples. Research like this is getting stronger and stronger. These recent studies are remarkable in their inclusion of many key fossils, and will become more valuable as the sample becomes more complete.

One practical barrier to including fossils may solved by greater investment: placing microCT facilities within the curating institutions. Where microCT facilities have been developed, they have unleashed research not only by increasing sample sizes but also by enabling greater leadership by researchers trained and employed in the countries where fossils are found. Every chance I get, I advocate to funding agencies and foundations to support the establishment of microCT facilities within the institutions that curate fossils. It is one of the most important practical steps that would broaden research.

At the same time, studies often suffer from artificial limits to their scope. Researchers who are studying the relationships of Homo habilis, for example, may choose to exclude the contemporary closely related species Paranthropus robustus and Paranthropus boisei from their study. In the work by Zanolli and coworkers, we see how such artificial limits could lead to misleading results: some of the so-called early Homo sample probably is Paranthropus. Each data point is vital. Methods should make the most of the fossil sample we have.

Another important methodological challenge is aligning the research on shape evolution with phylogenetic methods. Principal components plots are great visualizations for some datasets, but there's a limit on how much variation they can depict. When shape covariance can be aligned on two dimensions, PC plots are fine. But unless the evolution of shapes is tightly confined, two dimensions can really only depict the relationships of three groups. A fourth group with its own random evolutionary direction requires a third dimension. Phylogenetic methods provide some approaches to convey shape variation on trees of relationships.

Meanwhile studies of the enamel-dentin junction morphology are moving the ball forward. They're expanding samples and bringing light to some enigmatic fossils. More and more, this method is becoming a basis for identifying the distinctiveness of new species. In this case, they show that some species were less distinctive in their EDJ than other features had suggested.

Notes: I made some small edits on this after posting to address a sentence that left readers hanging. All fixed now.

The name Australopithecus habilis goes back to the 1960s, used by researchers who accepted the species but stuck to a more traditional definition of the genus Homo. While Robinson did not use this—he preferred to include the habilis fossils within africanus, and eventually came to the perspective that Several papers by Researchers who reconsidered habilis as part of Australopithecus included Bernard Wood and Mark Collard in a review in Science, as well as Milford Wolpoff in his 1999 textbook and other articles.

The story of South African fossil hominins has some fascinating details that are too involved for this post. One complicated element is the way that researchers have referred to the fossils that most now recognize as Homo. Initially, Robinson and Broom saw these as a distinct genus, Telanthropus, which they understood to be a close relative of Homo. Robinson abandoned this view by the 1960s, and by the 1970s he was attributing africanus to Homo also. But after the mid-1960s, many researchers would follow Tobias' ideas about the nature of Homo, accepting that some intermediate between Au. africanus and H. erectus was to be expected in South Africa, and looking for it in every fossil.

I have written several times about the varied opinions of different researchers about the taxonomy of Homo habilis and the definition of Homo. Readers who would like more background on these topics can see my Medium post, “The plot to kill Homo habilis” as well as my post here, “Should there be a new genus to contain Homo habilis?”

References

Clarke, R. J. (1977). A juvenile cranium and some adult teeth of early Homo from Swartkrans, Transvaal. South African Journal of Science, 73(2), 46. https://doi.org/10.10520/AJA00382353_7213

Davies, T. W., Gunz, P., Spoor, F., Alemseged, Z., Gidna, A., Hublin, J.-J., Kimbel, W. H., Kullmer, O., Plummer, W. P., Zanolli, C., & Skinner, M. M. (2024). Dental morphology in Homo habilis and its implications for the evolution of early Homo. Nature Communications, 15(1), Article 1. https://doi.org/10.1038/s41467-023-44375-9

Haeusler, M., & McHenry, H. M. (2004). Body proportions of Homo habilis reviewed. Journal of Human Evolution, 46(4), 433–465. https://doi.org/10.1016/j.jhevol.2004.01.004

Kimbel, W. H., Johanson, D. C., & Rak, Y. (1997). Systematic assessment of a maxilla of Homo from Hadar, Ethiopia. American Journal of Physical Anthropology, 103(2), 235–262. https://doi.org/10.1002/(SICI)1096-8644(199706)103:2<235::AID-AJPA8>3.0.CO;2-S

Leakey, L. S. B. (1966). Homo habilis, Homo erectus and the Australopithecines. Nature, 209(5030), Article 5030. https://doi.org/10.1038/2091279a0

Leakey, L. S. B., Tobias, P. V., & Napier, J. R. (1964). A New Species of The Genus Homo From Olduvai Gorge. Nature, 202(4927), Article 4927. https://doi.org/10.1038/202007a0

Moggi-Cecchi, J., Menter, C., Boccone, S., & Keyser, A. (2010). Early hominin dental remains from the Plio-Pleistocene site of Drimolen, South Africa. Journal of Human Evolution, 58(5), 374–405. https://doi.org/10.1016/j.jhevol.2010.01.006

Robinson, J. T. (1966). The Distinctiveness of Homo habilis. Nature, 209(5027), Article 5027. https://doi.org/10.1038/209957a0

Zanolli, C., Davies, T. W., Joannes-Boyau, R., Beaudet, A., Bruxelles, L., de Beer, F., Hoffman, J., Hublin, J.-J., Jakata, K., Kgasi, L., Kullmer, O., Macchiarelli, R., Pan, L., Schrenk, F., Santos, F., Stratford, D., Tawane, M., Thackeray, F., Xing, S., … Skinner, M. M. (2022). Dental data challenge the ubiquitous presence of Homo in the Cradle of Humankind. Proceedings of the National Academy of Sciences, 119(28), e2111212119. https://doi.org/10.1073/pnas.2111212119