cooperation

Dario Floreano and Laurent Keller describe experiments that combine genetic algorithms and robots. It's a review essay rather than a description of new research, but unlike most descriptions of "evolutionary robotics", it's actually directed toward biologists instead of AI researchers.

In this essay we will examine key experiments that illustrate how, for example, robots whose genes are translated into simple neural networks can evolve the ability to navigate, escape predators, coadapt brains and body morphologies, and cooperate. We present mostly—but not only—experimental results performed in our laboratory, which satisfy the following criteria. First, the experiments were at least partly carried out with real robots, allowing us to present a video showing the behaviours of the evolved robots. Second, the robot's neural networks had a simple architecture with no synaptic plasticity, no ontogenetic development, and no detailed modelling of ion channels and spike transmission. Third, the genomes were directly mapped into the neural network (i.e., no gene-to-gene interaction, time-dependent dynamics, or ontogenetic plasticity). By limiting our analysis to these studies we are able to highlight the strength of the process of Darwinian selection in comparable simple systems exposed to different environmental conditions.

Some of the simplest machine learning experiments are basically like those used in behavioral psychology -- put the robots in a maze, make them remember where the food is, that sort of thing. Robots are simpler than rats, so the researchers can reverse-engineer the "evolved" software at the end of a series of experiments to see what worked and why:

Interestingly, the driving speed of the best-evolved robots was approximately half of the maximum possible speed and did not increase even when the evolutionary experiments were continued for another 100 generations. Additional experiments where the speed was artificially increased revealed that fast-moving robots had high rates of collisions because the 300-ms refresh rate of the sensors did not allow them to detect walls sufficiently in advance at high speed. Thus, the robots evolved to move at intermediate speeds because of their limited neural and sensory abilities.

Figuring out that particular optimization would drive a team of human programmers crazy. Can you imagine? "Why do they keep running into that wall?!"

On the other hand, dumb selection took a lot of generations to get to that point. You can't say selection was more efficient. If you had a crew of programming grunts and forced them to sit in a room for 100 robot generations, they'd come up with something.

It's quite possible that a human would have come up with much better software, by pushing the robots past the limits of mutations on their "genomes". Selection has its own "sensor limitations", it can get stuck in a local optimum, and depends on the mutation structure to explore the landscape.

It helps if the landscape has some strong correlation structure. That's what came to my mind as I read their account of experiments to make robots cooperate:

However, when the arena contained both large and small tokens, the behaviour of robots was influenced by the group kin structure. In groups of unrelated robots (i.e., robots whose genomes where not more similar within than between groups), robots invariably specialised in pushing the small objects, which was the most efficient strategy to maximise their own individual fitness them (i.e., large tokens provided an equal direct payoff as a small token but were more difficult to successfully push). By contrast, the presence of related robots within groups allowed the evolution of altruism. When groups were formed of “clonal” robots all having the same genome, individuals primarily pushed the large tokens even though it was costly, in terms of individual fitness, for the robots pushing (Video S6).

If you wonder how robots have "kin", it's that they share similar (or the same) genomes. The simplicity of the behaviors suggests a functional explanation for kin selection -- for many kinds of tasks, it may simply be easier to cooperate with other individuals who "work" the same way. Different approaches to the same task may clash.

They describe a similar result for cooperation by information sharing:

Similar results were obtained in experiments where groups of light-emitting, foraging robots could communicate the position of a food source at a cost to themselves because of the resulting increased competition near food. In these experiments, robots again readily evolved costly communication when they were genetically related, but altruistic communication never evolved in groups of unrelated robots when selection operated at the individual level [38],[39].

The next logical step for this kind of research is nano-scale: evolutionary robotics on molecular machines. Which is scary. I hope they have the sense not to train them up by eating biological systems...

There's this old course on the books here, "Human aspects of robotics". I suppose it was taught back in the 80's when robots looked like they would replace all the manufacturing workers. I've often thought that someday it may be revived as with robots as the heroes instead of the villains.

References:

Floreano D, Keller L. 2010. Evolution of adaptive behavior in robots by means of Darwinian selection. PLoS Biol 8:e1000292. doi:10.1371/journal.pbio.1000292

I want to run through some examples of how we can apply game theory to consider hunting decisions in human groups. First, I describe a simple Snowdrift model applied to hunting. This is part 2 of a series, part 1 introduces the topic of the Snowdrift game.

A reader sent along a story after reading the first post:

In reading your snowdrift blog post, I was reminded of an experiment that does not require game theory to understand. You may have heard of it. Two pigs are in a pen. One is dominant. To get food one of them presses a bar, but the food is dispensed at the other side of the pen. If the subdominant pig presses the bar, it gets no reward, as the dominant pig hogs the food, eating it all. The result is that the dominant pig presses the bar while the subdominant pig waits at the food trough. Then the dominant pig rushes over to the trough to push the subdominant pig aside. Both pigs get fed, but the dominant pig does all the work

It's a great example of asymmetrical rewards. I'll get to those in the next few posts on this topic, because the asymmetries are very important to understanding dynamics in hunter-gatherers. But first, we have to describe the simple symmetrical case, including the algebra defining the evolutionarily stable equilibrium between the two simple strategies.

Suppose we have two hunters, who will share whatever game either of them kills. A man may choose on a given day to hunt. By hunting, he suffers a cost c and brings back a large fixed benefit b for each man. The two men may both choose to hunt on the same day, resulting in the same benefit b but a lowered cost c∕2 for each man. The two men decide whether to hunt simultaneously and without conferring — that is, there is no information transfer between them that would affect their decisions.

Here is the payoff matrix of the game for player 1 (choices on left) given the strategy selected by player 2 (on top):

Given the existence of the two strategies, “hunt” and “no hunt,” the ESS is the ratio at which the two strategies have equal expected returns. If individuals select a strategy and do not vary, the ESS represents the frequency of these variants in the population. If in contrast, individuals can choose to adopt either strategy, then the ESS also will be the optimal proportion of the two strategies in one individual’s repertoire. The two strategies will yield equal payoffs when the ESS satisfies the following equation, where p represents the proportion of “hunt” and 1 - p the proportion of “no hunt”:

(1)

…which simplifies to p = 2(b - c)∕(2b - c). That expression is positive where b > c, and approaches unity where c is very small relative to b. If in contrast b ≤ c then the scenario is the Prisoner’s Dilemma, where the only ESS is a pure “no hunt” strategy.

Let’s also look at a slightly different case. As above, each man’s return from hunting will be b regardless of whether one man or both choose to hunt. But in the payoff matrix below, the cost of hunting is also the same whether one man or both choose to hunt. So there is no reduction in the cost of hunting if both men do it.

Now, in this case, the ESS satisfies the equation:

(2)

Again, p is the frequency of the “hunt” strategy. This simplifies to p = (b-c)∕b, which again yields the Prisoner’s Dilemma when b < c.

OK, that’s the simple Snowdrift game model, described in the language of hunting instead of winter car accidents. It is quite simplistic in many ways. We might expect real hunters to have successes and costs that vary as stochastic functions of the environment. A real hunter must decide whether to hunt based not only on the odds his companion will hunt, but also upon some appraisal of the companion’s likelihood of success. Men in hunting societies are not paired up by the buddy system, but instead make their decisions about hunting in the context of a larger group’s activities.

Maybe most confusing, there are two possible kinds of currency in which benefits and costs may be expressed. A benefit from hunting may be most naturally measured in calories. If we average hunting returns across many episodes, then our result would be mean calories per day, or per hour of effort. Likewise, it might seem natural to discuss costs in terms of calories, as we might consider the cost of locomotion or cost of transport associated with foraging.

But the only currency that matters to evolution is fitness. We cannot assume that maximizing caloric returns will maximize fitness. Transport and locomotor costs may be minor compared to the mortality risk from predation when foraging far from camp. The caloric benefits from hunting matter more to a starving child than to a satiated adult.

So the measures of costs and benefits that define the ESS should be expressed in terms of fitness. That’s a problem, because fitness outcomes are a lot harder to measure than caloric returns. To figure out caloric expenditure and returns, you can measure oxygen consumption, work out distances, and weigh meat. To measure fitness, you have to record lifetime reproduction. To assess the relationship between caloric returns and fitness, you need a lifetime of caloric returns.

So far, hunter-gatherer demographic data and hunting returns are both known from a small number of transverse studies. Longitudinal data on hunter-gatherer demography are limited, and mostly known by retrospective methods — that is, informants share their knowledge about the history of their groups. The fitness effects of a single individual’s hunting effort over time are not known.

If fitness outcomes are hard for the scientist to measure, they are equally hard for a social actor to predict. Even intelligent actors like humans know little about the effects of their actions upon their future reproduction. Men sometimes do poorly with information directly relevant to fitness, like “Is the child mine?” That’s not to say that men may not follow highly sophisticated strategies to allocate hunting effort. But we should develop explanations that do not assume that a man knows the fitness benefits and costs of his choices.

Next: Life history and asymmetrical strategies

Sharon Begley in Newsweek reports on a hypothesis about "collectivism" and pathogens:

The West epitomizes individualistic, do-your-own thing cultures, ones where the rights of the individual equal and often trump those of the group and where differences are valued. East Asian societies exalt the larger society: behavior is constrained by social roles, conformity is prized, outsiders shunned. "The individualist-collectivist split is one of the most powerful differences among cultures," says Nisbett. But the reason a society falls where it does on the individualism-collectivism spectrum has been pretty much a mystery. Now a team of researchers has come up with a surprising explanation: disease-causing microbes. Societies that evolved in places with an abundance of pathogens, they argue, had to adopt behaviors that add up to collectivism, for reasons of sheer preservation. Societies that arose in places with fewer pathogens had the luxury of individualism, which is less effective at limiting the spread of disease but brings with it other social benefits, such as innovation.

There have been a lot of papers lately trying to match clines (that is, gradients) of phenotypes or genes with current ecological conditions. Climate is the most frequent (often, measured in terms of temperature or rainfall). Pathogens sort of follow the climate gradient, with some exceptions -- sometimes but not always allowing for the fact that malaria is the largest.

But in some cases, much more important will be the historical dimension. Many, (but not all) people who live in high-pathogen areas today had ancestors who adopted agriculture relatively early, began living in denser concentrations and larger groups, and who therefore experienced in common a range of selective pressures that have nothing to do with pathogens.

Would it be surprising that early agriculturalists living in emerging villages and cities might have been subject to pressures that enhanced collectivism? Such changes may have been facilitated by genetic changes, but would have also included cultural adaptations. Yet a correlation with pathogens would emerge as a side-effect of the history of agriculturalism, not as a direct cause.

To my mind, these kinds of historical correlations rooted in ecological change will be a central problem of anthropological genomics. Recent human evolution has been dominated by a few very large changes -- like boulders thrown into a pond that have spread massive ripples through many elements of human genetics, anatomy, and behavior. These changes are not yet complete -- the ripples have not settled down nor reached the shore. For this reason, there will be many correlations that have been induced by the large ecological changes, making bivariate spatial comparisons a poor test of cause.

I would say that historical correlation is a problem with a number of recent studies of cranial variation. Lately, it has been fashionable to test the hypothesis that cranial variation is adaptive to climate, by looking for spatial correlations between cranial measurements and climatic variables. But this test assumes that the correlations have not emerged as a result of some other cause. That might not be such a bad assumption, if humans had been static within their current geographic range for a long, long time. But humans have been anything but static -- in fact, their dynamism over the last 40,000 years has been the cause of profound changes in human biology. Naturally, things will be correlated with climate, because climate has been correlated with human subsistence and population size changes.

A better test of adaptation on cranial variables would propose concrete mechanical (or developmental) reasons why a cranial trait helps someone to better survive under a given climatic regime. Finding a correlation in space is not enough. This is why Bergmann's and Allen's rules are more compelling than the more nebulous idea that "facial form" adapts people to their climate.

Likewise in the case of pathogens -- a correlation between current pathogen load and current behavioral "collectivism" does not suggest a causal relationship between the two. Instead, we would want some kind of functional hypothesis to account for pathogens causing people to change in their attitudes toward cooperating. Concepts like the "luxury of individualism" make little sense. Of course, some people will prefer to behave with individual autonomy. But how would a recurrent epidemic disease stop them?