When the Society for Neuroscience gets together for their annual meeting each year, a city of scientists suddenly forms for a week. This year’s meeting has drawn 31,000 people to the Washington DC Convention Center. The subjects of their presentations range from brain scans of memories to the molecular details of disorders such as Parkinson’s and autism. This morning, a scientist named Svante Paabo delivered a talk. Its subject might make you think that he had stumbled into the wrong conference altogether. He delivered a lecture about Neanderthals.

Yet Paabo did not speak to an empty room. He stood before thousands of researchers in the main hall. His face was projected onto a dozen giant screens, as if he were opening for the Rolling Stones. When Paabo was done, the audience released a surging crest of applause. One neuroscientist I know, who was sitting somewhere in that huge room, sent me a one-word email as Paabo finished: “Amazing.”

You may well know about Paabo’s work. In August, Elizabeth Kolbert published a long profile in the New Yorker. But he’s been in the news for over fifteen years. Like many other journalists, I’ve followed his work since the mid-1990s, having written about pieces of Paabo’s work in newspapers, magazines, and books. But it was bracing to hear him bring together the scope of his research in a single hour–including new experiments that Paabo’s colleagues are presenting at the meeting. Simply put, Paabo has changed the way scientists study human evolution. Along with fossils, they can now study genomes that belonged to people who died 40,000 years ago. They can do experiments to see how some of those individual genes helped to make us human. During his talk, Paabo used this new research to sketch out a sweeping vision of how our ancestors evolved uniquely human brains as they swept out across the world.

Before the 1990s, scientists could only study the shape of fossils to learn about how we evolved. A million years ago, the fossil record contained evidence of human-like creatures in Europe, Asia, and Africa. Roughly speaking, the leading hypotheses for how those creatures became Homo sapiens came in two flavors. Some scientists argued that all the Old World hominins were a single species, with genes flowing from one population to another, and together they evolved into our species. Others argued that most hominin populations became extinct. A single population in Africa evolved into our species, and then later spread out across the Old World, replacing other species like Neanderthals in Europe.

It was also possible that the truth was somewhere in between these two extremes. After our species evolved in Africa, they might have come into contact with other species and interbred, allowing some DNA to flow into Homo sapiens. That flow might have been a trickle or a flood.

As scientists began to build a database of human DNA in the 1990s, it became possible to test these ideas with genes. In his talk, Paabo described how he and his colleagues managed to extract some fragments of DNA from a Neanderthal fossil–by coincidence, the very first Neanderthal discovered in 1857. The DNA was of a special sort. Along with the bulk of our genes, which are located in the nucleus of our cells, we also carry bits of DNA in jellybean-shaped structures called mitochondria. Since there are hundreds of mitochondria in each cell, it’s easier to grab fragments of mitochondrial DNA and assemble them into long sequences. Paabo and his colleagues used the mutations in the Neanderthal DNA, along with those in human and chimpanzee DNA, to draw a family tree. This tree splits into three branches. The ancestors of humans and Neanderthals branch off from the ancestors of chimpanzees 5-7 million years ago, and then humans and Neanderthals branch off in the last few hundred thousand years. If humans carried mitochondrial DNA from Neanderthals, you’d expect Paabo’s fossil genes to be more similar to some humans than others. But that’s not what he and his colleagues found.

Paabo and his colleagues then pushed forward and began to use new gene-sequencing technology to assemble a draft of the entire Neanderthal genome. They’ve gotten about 55% of the genome mapped, which is enough to address some of the big questions Paabo has in mind. One is the question of interbreeding. Paabo and his colleagues compared the Neanderthal genome to genomes of living people from Africa, Europe, Asia, and New Guinea. They discovered that people out of Africa share some mutations in common with Neanderthals that are not found in Africans. They concluded that humans and Neanderthals must have interbred after our species expanded from Africa, and that about 2.5% of the genomes of living non-Africans comes from Neanderthals.

This pattern could have arisen in other ways, Paabo granted. The ancestors of Neanderthals are believed to have emerged from Africa hundreds of thousands of years ago and spread into Europe. Perhaps the humans who expanded out of Africa came from the birthplace of Neanderthals, and carried Neanderthal-like genes with them.

But Paabo doubts this is the case. One way to test these alternatives is to look at the arrangement of our DNA. Imagine that a human mother and Neanderthal father have a hybrid daughter. She has two copies of each chromosome, one from each species. As her own eggs develop, however, the chromosome pairs swap some segments. She then has children with a human man, who contributes his own human DNA. In her children, the Neanderthal DNA no longer runs the entire length of chromosomes. It forms shorter chunks. Her children then have children; her grandchildren have even shorter chunks.

Paabo described how David Reich of Harvard and other scientists measured the size of the chunks of Neanderthal DNA in people’s genomes. They found that in some of the Europeans they studied, the Neanderthal chunks were quite long. Based on their size, the scientists estimated that the interbreeding happened between 37,000 and 86,000 years ago. (This research is still unpublished, but Reich discussed it at a meeting this summer.)

The success with the Neanderthal genome led Paabo to look for other hominin fossils that he could grind up for DNA. DNA probably can’t last more than a few hundred thousand years before degrading beyond recognition, but even in that window of time, there are plenty of interesting fossils to investigate. Paabo hit the jackpot with a tiny chip from the tip of a 40,000-year-old pinky bone that was found in a Siberian cave called Denisova. The DNA was not human, nor Neanderthal. Instead, it belonged to a distant cousin of Neanderthals. And when Paabo and his colleagues compared the Denisovan DNA to human genomes, they found some Denisovan genes in the DNA of their New Guinea subject. Mark Stoneking, Paabo’s colleague at Max Planck, and other scientists have expanded the comparison and found Denisovan DNA in people in Australia and southeast Asia.

Paabo then offered a scenario for human evolution: about 800,000 years ago, the ancestors of Neanderthals and Denisovans diverged from our own ancestors. They expanded out of Africa, and the Neanderthals swept to the west into Europe and the Denisovans headed into East Asia. Paabo put the date of their split about 600,000 years ago. The exact ranges of Neanderthal and Denisovans remain fuzzy, but they definitely lived in Denisova at about the same time 50,000 years ago, given that both hominins left bones in the same cave.

Later, our own species evolved in Africa and spread out across that continent. Humans expanded out of Africa around 100,000 years ago, Paabo proposed. (I’m not sure why he gave that age, instead of a more recent one.) Somewhere in the Middle East, humans and Neanderthals interbred. As humans continued to expand into Europe and Asia, they took Neanderthal DNA with them. When humans got to southeast Asia, they mated with Denisovans, and this second addition of exotic DNA spread through the human population as it expanded. Neanderthals and Denisovans then became extinct, but their DNA lives on in our bodies. And Paabo wouldn’t be surprised if more extinct hominins turn out to have donated DNA of their own to us.

Paabo sees these results as supporting the replacement model I described earlier–or, rather, a “leaky replacement” model. If humans and other hominins had been having lots of sex and lots of kids, we’d have lots more archaic DNA in our genomes.

Now that scientists know more about the history of our genome, they can start tracking individual genes. When I first wrote about this interbreeding work last year for the New York Times, I asked Paabo if there were any genes that humans picked up from interbreeding that made any big biological difference. He didn’t see any evidence for them at the time. But at the meeting, he pointed to a new study of immune genes. One immune gene appears to have spread to high frequency in some populations of Europeans and Asians, perhaps because it provided some kind of disease resistance that benefited them.

The history of other genes is just as interesting. Some of our genes have mutations also found in Neanderthals and Denisovans, but not in chimpanzees. They must have evolved into their current form between 5 million and 800,000 years ago. Other genes have mutations that are found only in the human genome, but not in those of Neanderthals and Denisovans. Paabo doesn’t have a complete list yet, since he’s only mapped half the Neanderthal genome, but the research so far suggests that the list of new features in the human genome will be short. There are only 78 unique human mutations that changed the structure of a protein. Paabo can’t yet say what these mutations did to our ancestors. Some of the mutations alter the address labels of proteins, for example, which let cells know where to deliver a protein once they’re created. Paabo and his colleagues have found that the Neanderthal and human versions of address labels don’t change the delivery.

Other experiments Paabo and his colleagues have been running have offered more promising results. At the talk, Paabo described some of his latest work on a gene called FoxP2. Ten years ago, psychologists discovered that mutations to this gene can make it difficult for people to speak and understand language. (Here’s a ten-year retrospective on FoxP2 I wrote last month in Discover.) Paabo and his colleagues have found that FoxP2 underwent a dramatic evolutionary change in our lineage. Most mammals have a practically identical version of the protein, but ours has two different amino acids (the building blocks of proteins).

The fact that humans are the only living animals capable of full-blown language, and the fact that this powerful language-linked gene evolved in the human lineage naturally fuels the imagination. Adding fuel to the fire, Paabo pointed out that both Neanderthals and Denisovans had the human version of FoxP2. If Neanderthals could talk, it would be intriguing that they apparently couldn’t paint or make sculptures or do other kinds of abstract expressions that humans did. And if Neanderthal’s couldn’t talk, it would be intriguing that they already had a human version of FoxP2. As scientific mysteries go, it’s a win-win.

From a purely scientific point of view, the best way to investigate the evolution of FoxP2 would be to genetically engineer a human with a chimpanzee version of the gene and a chimpanzee with a human version. But since that’s not going to happen anywhere beyond the Island of Doctor Moreau, Paabo is doing the second-best experiment. He and his colleagues are putting the human version of FoxP2 into mice.

The humanized mice don’t talk, alas. But they do change in many intriguing ways. The frequency of their ultrasonic squeaks changes. They become more cautious about exploring new places. Many of the most interesting changes happen in the brain. As I wrote in my Discover column, Paabo and his colleagues have found changes in a region deep in the brain called the striatum. The striatum is part of a circuit that lets us learn how to do new things, and then to turn what we learn into automatic habits. A human version of FoxP2 makes neurons in the mouse striatum sprout more branches, and those branches become longer.

Paabo’s new experiments are uncovering more details about how human FoxP2 changes the mice. Of the two mutations that changed during human evolution, only one makes a difference to how the striatum behaves. And while that difference may not allow mice to recite Chaucer, they do change the way they learn. Scientists at MIT, working with Paabo, have put his mice into mazes to see how quickly they learn how to find food. Mice with human FoxP2 develop new habits faster than ones with the ordinary version of the gene.

So for now, Paabo’s hypothesis is that a single mutation to FoxP2 rewired learning circuits in the brain of hominins over 800,000 years ago. Our ancestors were able to go from practice to expertise faster than earlier hominins. At some point after the evolution of human-like FoxP2, our ancestors were able to use this fast learning to develop the quick, precise motor control required in our lips and tongues in order to speak.

I think what made Paabo’s talk so powerful for the audience was that he was coming from a different world–a world of fossils and stone tools–but he could talk in the language of neuroscience. As big as the Society for Neuroscience meetings can be, Paabo showed that it was part of a much bigger scientific undertaking: figuring out how we came to be the way we are.

[Image: Frank Vinken]

Originally published November 14, 2011. Copyright 2011 Carl Zimmer.