Multi-Brain HyperNEAT Experiments in the Five Environments domain
Multi-Brain HyperNEAT
is a software framework that extends
HyperNEAT
to evolve agents that possess multiple brains. Different brains can
be used in different situations, making it easy to evolve multimodal
behavior.
The five environments domain combines the previously studied
team patrol,
lone patrol,
dual task, and
two rooms domains.
This list of four domains actually comprises five distinct
environments, because dual task has both a hallway/navigation
task and a foraging task. A single evolved CPPN must generate
agents to perform in all five environments.
The videos on this page can be viewed in a playlist here.
Five Environments: One Module (1M) - Typical Champion
Most champions with only one module do not perform well, which is why this approach has the lowest overall performance compared to methods with multiple brains. This champion has average performance among 1M results. It does not do well in team patrol or two rooms (in contrast to high-scoring champions of other methods). Its lone patrol performance is decent, and its performance in the two dual task environments is perfect (but this is fairly common). Full scores below.
Total Average: 0.601674454369932
Team Patrol: 0.0436650178763011
Lone Patrol: 0.682556881110544
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 0.282150372862816
Five Environments: One Module (1M) - Best of 30 Champions
This is the best evolved champion with one module in the five environments domain. It's behavior is not great, and it is an exception among its peers. The reason this agent earns the score it does is that it solves both environments from the dual task, and also earns an almost perfect score in Two Rooms. It's score in Lone Patrol is close to the best scores earned by other methods, but its performance in Team Patrol is terrible. However, it should be noted that even the best multimodal methods have trouble doing well in both Team Patrol and Two Rooms. The final scores of this agent are shown below.
Average score: 0.746155734795519
Team Patrol: 0.0361431325378699
Lone Patrol: 0.776652911008074
Dual Task Hall: 1
Dual Task Foraging: 1
Two Rooms: 0.917982630431652
Five Environments: Two Modules (2M)
Champions with two preference modules do better than 1M because they can start to make use of multiple modules. This champion performs poorly in Team Patrol, but masters both Dual Task environments and Two Rooms. Performance in Lone Patrol is decent. Interestingly, only one (green) module is used in both patrol environments. The other (red) module is used in the other environments, and the Hallway and Two Rooms environments actually use both modules. Specifically, the green module helps avoid walls in tight spaces. Full scores below.
Total Average: 0.738567683253416
Team Patrol: 0.0293565577380436
Lone Patrol: 0.663481858529036
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 1
Five Environments: Three Modules (3M)
This agent with three preference modules earns a very high average score because, despite not mastering the Team Patrol or Two Rooms environments, it does perform fairly well in both, which distinguishes it from weaker 1M and 2M agents that master one of these two domains while performing terribly in the other. In fact, this particular champion is even exceptional compared to many 4M and 5M champions. This 3M champion does master both dual task environments, and performs well in Lone Patrol too. Scores are below:
Total average: 0.852459820380177
Teal Patrol: 0.835407682834051
Lone Patrol: 0.69765817649548
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 0.729233242571354
Five Environments: Four Modules (4M)
This four module champion uses several modules and earns high scores in both patrol environments. It also earns perfect scores in the dual task environments. The only environment where it performs poorly is the two rooms environment. Although performance is not perfect in the patrol environments, modules are clearly being used to clear effect. The same is true in the other environments. Full scores below:
Total Average: 0.773701667710552
Team Patrol: 0.839808856812029
Lone Patrol: 0.746549108877917
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 0.282150372862816
Five Environments: Five Modules (5M) - Skilled at Team Patrol
Agents with more preference modules do better on average than those with fewer, but even the champions of many runs will sacrifice behavior in some environments to do better in others. The primary talent of this 5M champion is its performance in Team Patrol, which results in two of the three agents returning home. It also performs fairly well in Lone Patrol and masters the Dual Task Foraging environment, but it actually fails the Dual Task Hallway environment and also performs poorly in the Two Rooms environment. It's final scores are below:
Total Average: 0.735689684232609
Team Patrol: 0.942077222864731
Lone Patrol: 0.689496174440542
Dual Task Hallway: 0.764724650994954
Dual Task Foraging: 1
Two Rooms: 0.282150372862816
Five Environments: Five Modules (5M) - Skilled at Two Rooms
This champion with five preference modules performs poorly in Team Patrol, but well in the other domains. In particular, it masters both dual task environments and Two Rooms. Interestingly, green and red modules are used in all three of these environments. In contrast, a blue module is used in the patrol environments only. Performance in Lone Patrol is reasonably good, though this blue module is clearly not helping in Team Patrol, which uses a mix of red and blue. Full scores below:
Total Average: 0.770919157141096
Team Patrol: 0.0639856943002679
Lone Patrol: 0.790610091405211
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 1
Five Environments: Multitask (MT)
Despite having a separate brain dedicated to each environment, Multitask champions do not actually perform very well, reaching about the same level of performance as two preference modules (2M). Also, in contrast to preference-based champions, which typically do very well in either Two Rooms or Team Patrol at the expense of the other, Multitask champions just earn mediocre scores in both. No Multitask champion masters Two Rooms (in contrast, despite having worse average scores overall, there are at least a few 1M champions that master Two Rooms). Full scores of this agent are below:
Total Average: 0.680828062592745
Team Patrol: 0.0487421938839924
Lone Patrol: 0.698247746216919
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 0.657150372862816
Five Environments: Situational Policy Geometry (SPG)
Situational Policy Geometry performs particularly poorly in the Five Environments domain because there is no geometric relationship between these environments. Most of these champions earn scores below 0.6, but this exception is better than the rest. It actually performs well in Team Patrol, and masters both Dual Task environments. Full scores below:
Total Average: 0.774143110743996
Team Patrol: 0.846515727739622
Lone Patrol: 0.679549453117542
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 0.344650372862816
Five Environments: Module Mutation Duplicate (MM(D))
This particular MM(D) champion has eight modules and uses five, allowing it to earn perfect scores in both dual task environments, and very high scores in both patrol environments. The two rooms environment is the only one in which it performs poorly. Two modules (tan and dark tan) are used only in team patrol, which is also the only environment in which the red module is not used. The blue/cyan module is used in all environments, but most heavily in the patrol environments. Although the red module is used in lone patrol, it is mostly used in the dual task and two rooms environments. These usage patterns indicate both division of responsibilities and specialization for particular behaviors. Final scores below.
Total Average: 0.780523190076459
Team Patrol: 0.841369176525947
Lone Patrol: 0.779096400993534
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 0.282150372862816
Five Environments: Module Mutation Previous (MM(P))
This particular MM(P) champion has eight modules, but only uses three. However, the only environment it performs poorly in is team patrol. It earns perfect scores in both dual task environments and two rooms using only the tan and blue/cyan modules. Lone patrol performance is not perfect, but is fairly good, and depends on rare use of a third module: green. This green module is actually used exclusively by two of the team members in the team patrol environment to disastrous effect: all team members head to the same patrol point, and none of them return. Final scores below.
Total Average: 0.740619650071198
Team Patrol: 0.0311271863961145
Lone Patrol: 0.671971063959873
Dual Task Hallway: 1
Dual Task Foraging: 1
Two Rooms: 1
Five Environments: Module Mutation Random (MM(R))
This MM(R) champion creates many unused modules, but only uses five. The dual task environments and two rooms make exclusive use of the green and cyan modules. This leads to success in dual task foraging and failure in two rooms, neither of which is very surprising, but the champion is also unsuccessful in dual task hallway, which is a bit unusual. Nevertheless, a decent average score is earned due to good performance in lone patrol (using modules blue, green, and a little bit of red), and great (though not perfect) performance in team patrol. Module usage in team control is extremely complex and involves very different module usage patterns by the three team mates: One primarily uses cyan, with a bit of red and even less green; one uses white mixed with a bit of green and red, and the third uses green and blue. Only the second of these successfully returns home, but all make a decent attempt and at least reach the center before either getting stuck or choosing the wrong route. Final scores below.
Total average: 0.684154132101999
Team patrol: 0.847093773513427
Lone Patrol: 0.665955661013279
Dual Task Hallway: 0.625570853120475
Dual Task Foraging: 1
Two Rooms: 0.282150372862816