Cultural Evolution and Alignment

transcript

One of the things that I work on is the evolution of the brain. So one question that you might take for granted is: why do humans have such large brains? You should be puzzled by this because you might wonder, well, why don't other animals have large brains? In general, there's negative selection for bigger brains because bigger brains allow you to process more information and do amazing things, but they come with a massive calorie cost. For most animals, it means requiring food constantly. You're always looking for food. Humans in evolving our large brains managed to offset some of that in a variety of ways. One is that we shrunk our guts. We learned to control fire and we learned to have cooked food, which made calories more bioavailable.

So the human brain starts off about the same size as a chimp brain and then triples in size over the last few million years. We separate from the last common ancestor about four to seven million years ago. And then you see this massive rapid uptake with our lineage.

Then most recently, in the last three to ten thousand years — the exact time period is disputed — we actually see a shrinkage in the brain. So over the Holocene, 12,000 years ago, all of our ancestors are hunter-gatherers. And then around 10,000 years ago, the climate changes. We do shrink in body size, but our brains seem to shrink more than allometry would predict, so it's not just a shrinkage with body size. With the advent of agriculture, it seems like our brains shrank. And just to be very clear, it's not like a small shrinkage — it's about the size of a tennis ball.

The reason that there's negative selection is that brain tissue is about 20 times as expensive as muscle tissue. So it really has to pay its calorie costs. For most animals, when they encounter a new environment, they invest in brawn — being faster, stronger, to evade predators and seek out prey. But it's also true that all else being equal, larger brains allow you to store and manage more information. This is true between species within the same taxa. It's also true within species — even among humans, the correlation between brain size and IQ is about 0.2 to 0.3, depending on how you measure it.

For cultural evolution and for us, we see a whole bunch of skills begin to emerge as we switch from the two lines of information that most animals have — their genetic instincts and what they can learn over their lifetime — to learning from other members of our group. This is the process that we call cultural evolution. And once we switch to that, as the corpus of information grows, this creates a massive selection pressure. The data starts to grow, and so the compute requirements also grow. We're learning to use tools. We're learning to control fire. The fire allows us to reliably have cooked food. The cooked food allows us to shrink our guts, shrink our jaws, and then grow the brains.

As this begins to grow, we get a whole bunch of psychological traits. Our theory of mind improves so that we can better learn from one another. Our gaze following — we tend to focus on what other people do. And more importantly, we start to defer to the group. We no longer need to understand everything that's going on. We just copy what other people are doing. You snap the flint in a particular way, you're going to get a tool. You strike it a particular way, you get a fire.

When agriculture emerges, hunter-gatherers are in many ways more empowered than we are in that they have to know everything it takes to survive. There's much less division of labor. You have to know how to build a shelter, how to make a fire and cook food, how to process that food, what the laws are — you need to know it all. But when agriculture emerges and our group sizes expand significantly, and we have a more reliable source of food, suddenly we can begin to divide up the knowledge. If you imagine each of us has a size-ten brain and before you had to allocate it to ten items, you could reach skill unit one. But if you can reliably learn only 50% of that, you can reach skill unit two. As we became more divided in our labor, the more we could specialize, the more as a group humanity is just clever. But that disempowers us individually.

Today we know very little about how the world works. Each of us is super, super smart in one thing and kind of a moron about everything else. We don't really understand our food chains. We don't understand how the chair that you're sitting on is made. A kind of illusion of explanatory depth. So what we think might be going on here is this gradual disempowerment means that the requirement for a very large brain, at least at first, decreases. And so you get selection for a smaller brain.

The overall point is that the gradual disempowerment of the kind that we're describing — where suddenly we don't really understand how the world works and we're reliant on the machine, where the machine is the rest of humanity — it's happened already, and it had implications for us.

With human groups, religion is probably a really nice example of this. Human groups are competing with one another on sets of moral norms, values, ideas. And if a lot of these moral norms just seem obviously true and obviously correct to you, I think their "common sense" is because you're on the winning side of this cultural evolutionary process. People have often noted that religions seem to have a lot of common values — be nice to other people, an emphasis on family and looking after one another. It's not the case across the historical record that every single religion had these traits.

My favorite example is: many of you probably know some Quakers. Do you guys know any Shakers? Unlikely — but you know who they are. The Shakers were an offshoot of the Quakers who believed in celibacy not for the priestly class, but for everybody. The only way they grew was through adoption and conversion. There are no Shakers left. Certain traits are just not winning traits, and over time you get this winnowing process where particular sets of norms, particular social configurations, win out against others. And in highly competitive group-level situations — like in Europe with agriculturalists fighting against other agriculturalists — you get these kind of group-beneficial norms, what you might call morality, start to dominate.

So what happened when those agriculturalists met hunter-gatherers? What happened was marginalization. Today, hunter-gatherer groups live in two kinds of places: deserts and thick forests. The reason is that they were marginalized to the environments that weren't useful for agriculture. Hunter-gatherers either converted to become agriculturalists — it's not like we created new humans — and those who were left behind practicing traditional ways of life were marginalized by this smarter and more sophisticated civilization.

The second scenario is what happened when we reached another threshold. The first major threshold was probably fire. The second was agriculture. And the next was the industrial revolution. Most of the psychology we have today, most of the moral norms we have today, are in the shadow of the industrial revolution. We discovered a bunch of stored sunlight that had been pressure-cooked for millions of years into useful batteries. We learned how to put those into work. And we created a positive-sum world in which it paid to cooperate, to work together in guilds, to work together as a group and colonize less cooperative pre-industrial agricultural societies.

In this second case, something slightly different happened. Agriculturalists needed the land, but they didn't necessarily need the labor — hunter-gatherers were low productivity and there's not a lot of them anyway. But in the industrial case, what we saw was exploitation. You still need food. And so those agricultural societies end up in lower economic niches. They're making your iPhones. They're growing your food. They're slotted into subordinate economic niches. This is a categorically different outcome from marginalization. But it depends on the same thing: what does the dominant civilization actually want?

Human wants have been pre-trained by millions of years of evolution. Often what we want is the proximate thing that used to get us the ultimate fitness thing. It's not that we want children — it's that we want sex. It's not that we necessarily want fitness — what we want is prestige. It's not that we want nutrition — we want fat, we want protein, we want sweet things. Evolution has tuned us to the proxies because in the evolutionary environment, there was no disentangling between the proxy and the thing that mattered. Today there is. We have the same paperclip problem in the sense that because we're tuned for sweet things and now there's an abundance of them, we see obesity. Because we're tuned for sex and not children, there's a fertility crisis. We're over-optimizing this. But these don't come out of nowhere. They come out of a reinforcement learning process, like a control system, whereby the organisms that enjoyed sex left offspring and those that preferred banging their head against the wall, like the Shakers, are no longer with us. So there's a tuning of these preferences.

Beyond those genetic preferences, we also have norms, which are also selected by a competitive process. There are norms around marriage practices — you have dowry systems, you have bride prices, you have polygynous systems where you have multiple wives, which is the most common marital system on the planet. You get monogamous systems, largely spread by Christianity. And in some cases you get polyandrous situations, typically in places where one male can't provision their children, and then typically it's brothers, so there's at least genetic relatedness. We have norms around status, around food, around fairness, around punishment and reputation. Again, these don't come out of nowhere — they come out of this selective process. So that's what humans want. But what about AGI? There are some basics. Any AGI that has traits that allow it to survive and spread will outcompete the AGI that don't have those traits. That's just a universal truism. What does an AGI need? It needs energy, compute, resources. So if we look like the hunter-gatherers, maybe we end up in the places that aren't as suitable — because they don't have a heat sink, they don't have good sunlight for energy. Beyond that, what the AGI wants is either engineered or selected for. But again, if it is going to be selected for, a lot of what happened for the higher-order scales of human cooperation happened not at an individual selective level, which often selects for more selfish traits — it happened at a group level. Groups that were able to coordinate survived at the expense of groups that didn't.

But there are other things as well. The skills that were valuable for humans have shifted as our technology has shifted. Writing and printing — Aristotle said writing is going to destroy human memory. He was really concerned: people are going to write things down and learn from this writing, and it's going to be fundamentally different to learning from another human being, and this is a really bad thing. And it's kind of true — I make lists that tell me what to do, I follow a GTD strategy or something. So there's less of a need to memorize the holy books, to memorize every aspect of law, to memorize everything about how to survive, because I can go look that up. That selection pressure has been culturally reduced. Agriculture and cities reduced the need for raw individual compute. But what about when we have AGI? What selection does that reduce? We actually don't know. All of our technologies thus far have enhanced our muscle or our computation a little bit. But we've never outsourced the thinking before.

The first implication is that AI-human collaboration, in the way that it happens with great-power proxy wars today — humans might retain some niche value if there are competing AGIs, where some of them at least might find humans useful. A second niche that we might have is that humans might be energy-cheaper than their AGI equivalent. Our brains run at about 20 watts. In the same way that an AGI might use a smaller model for certain things, the fact that we have senses and an ability to compute certain things — some tasks might be put to humans. Third, superintelligence might not be enough. A lot of people come from the world of software, which is one of the frictionless spaces where ideas are quickly optimized and you can move fast. In the real world, there's way more friction. It's not that people don't know what to do — it's that there are entrenched norms that prevent it from happening.

The fourth insight is that alignment and norms are a function of the selection regime, not necessarily the individual mind or the agent. In one scenario, the selection regime maintains a kind of human substrate or scaffolding — humans are by design in the loop. This might require intervention on the demand side, by institutions, markets, governance, rather than on the model side, so the payoff function includes us. In this scenario, it would be similar to hunter-gatherers becoming agriculturalists — we're part of the system.

The second scenario is that we actually try to engineer or evolve a cultural overlay equivalent — selecting at a higher level of groups of humans or groups of AI working together. There's a fan theory that Cameron's Avatar is actually a post-AGI world — that's why everyone has USB sticks that you can plug into, the Na'vi don't understand how the world works, the world works on its own, and they just occupy this niche that the AI doesn't really require.