Modular Multiobjective (Hyper) NEAT Source Code

Modular Multiobjective (Hyper) NEAT, or simply MM-NEAT, is a Java-based framework that extends NEAT to use the multiobjective optimization algorithm Non-dominated Sorting Genetic Algorithm II (NSGA-II) to evolve agents that effectively possess multiple brains. Different brain modules can be used in different situations, making it easy to evolve multimodal behavior. This project began as part of Dr. Jacob Schrum's dissertation, put has since been expanded upon to include an implementation of HyperNEAT, and also support several forms of interactive evolution. MM-NEAT also has many interesting domains integrated into the code: Ms. Pac-Man, Tetris, a Predator/Prey grid world, and Doom to name a few. Support for Latent Variable Evolution to generate video game levels with a Generative Adversarial Network has also recently been added. The Java source code is available via GitHub here. This version of the code uses Maven for dependency management, and continues to be developed. However, because Maven is sometimes intimidating to those unfamiliar with it, several spin-offs of the main MM-NEAT repository have been made, such as GameGAN, which is for latent variable evolution of video game levels, Quantum Zentanglement, which combines Picbreeder and WaveFunctionCollapse to generate Zentangles, and CPPN Art Evolution, which has a variety of systems for interactive art generation via Compositional Pattern Producing Networks.

Associated Movies and Images

Associated Publications

Dissertation

Jacob Schrum (2014). Evolving Multimodal Behavior Through Modular Multiobjective Neuroevolution, PhD Thesis, The University of Texas at Austin, May 2014. Tech Report TR-14-07. (slides, movies 1, movies 2, software).

Peer-Reviewed Journal Articles

Jacob Schrum, and Risto Miikkulainen (2016). Solving Multiple Isolated, Interleaved, and Blended Tasks through Modular Neuroevolution, Evolutionary Computation Journal (ECJ 2016). September. Volume 24, No. 3, pages 459 - 490. MIT Press. (blended version movies, interleaved version movies, isolated version movies, software).

Jacob Schrum, and Risto Miikkulainen (2016). Discovering Multimodal Behavior in Ms. Pac-Man through Evolution of Modular Neural Networks, IEEE Transactions on Computational Intelligence and AI in Games (TCIAIG 2016). March. Volume 8, Issue 1, pages 67 - 81. IEEE. (movies, software).

Peer-Reviewed Conference Publications

Benjamin Capps, and Jacob Schrum (2021). Using Multiple Generative Adversarial Networks to Build Better-Connected Levels for Mega Man, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2021). (presentation, slides, levels, software).

Jacob Schrum, Vanessa Volz, and Sebastian Risi (2020). CPPN2GAN: Combining Compositional Pattern Producing Networks and GANs for Large-scale Pattern Generation, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2020). (presentation, slides, software).

Jacob Schrum, Jake Gutierrez, Vanessa Volz, Jialin Liu, Simon M. Lucas, and Sebastian Risi (2020). Interactive Evolution and Exploration Within Latent Level-Design Space of Generative Adversarial Networks, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2020). (presentation, slides, movie, software).

Jake Gutierrez, and Jacob Schrum (2020). Generative Adversarial Network Rooms in Generative Graph Grammar Dungeons for The Legend of Zelda, Proceedings of the Congress on Evolutionary Computation (CEC 2020). (presentation, slides, movies, software).

Anna Krolikowski, Sarah Friday, Alice Quintanilla, and Jacob Schrum (2020). Quantum Zentanglement: Combining Picbreeder and Wave Function Collapse to Create Zentangles, EvoMUSART: International Conference on Computational Intelligence in Music, Sound, Art, and Design. (presentation, slides, art, software).

Adina Friedman, and Jacob Schrum (2019). Desirable Behaviors for Companion Bots in First-Person Shooters, Proceedings of the Conference on Games (CoG 2019). (slides, movies, software).

William Price, and Jacob Schrum (2019). Neuroevolution of Multimodal Ms. Pac-Man Controllers Under Partially Observable Conditions, Proceedings of the Congress on Evolutionary Computation (CEC 2019). (slides, movies, contest entry, evolution software).

Jacob Schrum, (2018). Evolving Indirectly Encoded Convolutional Neural Networks to Play Tetris With Low-Level Features, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2018). (slides, movies, software).

Isabel Tweraser, Lauren E. Gillespie, and Jacob Schrum (2018). Querying Across Time to Interactively Evolve Animations, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2018). (slides, art and movies, software).

Vanessa Volz, Jacob Schrum, Jialin Liu, Simon M. Lucas, Adam M. Smith, and Sebastian Risi (2018). Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2018). (slides, explanation movie, evolution movie, software). Best Paper Award in Digital Entertainment Technologies and Arts

Lauren E. Gillespie, Gabriela R. Gonzalez, and Jacob Schrum (2017). Comparing Direct and Indirect Encodings Using Both Raw and Hand-Designed Features in Tetris, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2017). (slides, movies, software).

Jacob Schrum, and Risto Miikkulainen (2015). Solving Interleaved and Blended Sequential Decision-Making Problems through Modular Neuroevolution, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2015). (slides, interleaved domain movies, blended domain movies, software). Best Paper Award in Digital Entertainment Technologies and Arts (also extended into a journal article)

Jacob Schrum, and Risto Miikkulainen (2014). Evolving Multimodal Behavior With Modular Neural Networks in Ms. Pac-Man, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2014). (slides, movies, software). Best Paper Award in Digital Entertainment Technologies and Arts (also extended into a journal article)

Extended Abstracts

Kirby Steckel, and Jacob Schrum (2021). Illuminating the Space of Beatable Lode Runner Levels Produced By Various Generative Adversarial Networks, Companion to the Proceedings of the Conference on Genetic and Evolutionary Computation (GECCO Companion 2021). (poster, levels, software). (also extended into a technical report)

Alex Rollins, and Jacob Schrum (2017). Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior, Companion to the Proceedings of the Conference on Genetic and Evolutionary Computation (GECCO Companion 2017). (poster, movies, software). (also extended into a technical report)

Technical Reports

Kirby Steckel, and Jacob Schrum (2021). Illuminating the Space of Beatable Lode Runner Levels Produced By Various Generative Adversarial Networks, ArXiv e-print: 2101.07868, January, 2021. (levels, software).

Alex Rollins, and Jacob Schrum (2017). Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior, ArXiv e-print: 1703.08577, March, 2017. (movies, software).

Undergraduate Poster Presentations Supervised

Benjamin Capps (2020). Evolving Mega Man Levels Using Generative Adversarial Networks, Southwestern University Science Research Open House. (levels, software).

Kirby Steckel (2020). Evolving Lode Runner Levels with Generative Adversarial Networks, Southwestern University Science Research Open House. (levels, software).

Ryan Wheeler, Nicholai Benko, and Guy Greathouse (2020). Using Neuroevolution on Checkers Players, Alpha Chi National Convention. (software).

Jake Gutierrez (2019). Creating Zelda Dungeons with a Graph Grammar and a Generative Adversarial Network, Southwestern University Science Research Open House. (movies, software).

Devon Fulcher (2019). Evolving Custom Convolutional Neural Network Architectures in SZ-Tetris, South Central Regional Meeting of the Consortium for Computing Sciences in Colleges. (software). 4th Place: Undergraduate Poster Competition

Adina Friedman (2019). Desirable Behaviors for Companion Bots in First-Person Shooters, South Central Regional Meeting of the Consortium for Computing Sciences in Colleges. (movies, software).

William Price (2018). Evolving an Intelligent Ms. Pac-Man Agent Under Partially Observable Conditions, Southwestern University Science Research Open House. (movies, contest entry, evolution software).

Devon Fulcher (2018). Evolving Custom Convolutional Neural Network Architectures in SZ-Tetris, Southwestern University Science Research Open House. (software).

Adina Friedman (2018). Creating Evolved and Hand Coded Companion Characters, Southwestern University Science Research Open House. (movies, software).

Sarah "Darwin" Johnson (2018). Evolution of Board Game Playing Agents, South Central Regional Meeting of the Consortium for Computing Sciences in Colleges. (software).

Alice Quintanilla (2018). Evolving Artificial Intelligences to Compete in Real-Time Strategy Games, South Central Regional Meeting of the Consortium for Computing Sciences in Colleges. (movies, software).

Sarah "Darwin" Johnson (2017). Evolution of Board Game Playing Agents, Southwestern University Science Research Open House. (software).

Alice Quintanilla (2017). Evolving Artificial Intelligences to Compete in Real-Time Strategy Games, Southwestern University Science Research Open House. (movies, software).

Isabel Tweraser (2017). Using Time to Interactively Evolve Animations, Southwestern University Science Research Open House. (movies, software).

Isabel Tweraser (2017). Breedesizer: Evolving Original Sounds With Neural Networks, Southwestern University Science Research Open House. (software).

Lauren Gillespie, and Gabriela Gonzalez (2017). Comparing Direct and Indirect Encodings Using Both Raw and Hand-Designed Features in Tetris, South Central Regional Meeting of the Consortium for Computing Sciences in Colleges. (movies, software). 1st Place: Undergraduate Poster Competition

Alex Rollins (2017). Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior, South Central Regional Meeting of the Consortium for Computing Sciences in Colleges. (movies, software). 2nd Place: Undergraduate Poster Competition

Gabriela Gonzalez, and Lauren Gillespie (2017). Evolution of Artificial Intelligence in Tetris Play Using Raw Screen Inputs, Annual Meeting of the Texas Academy of Sciences. (movies, software). Best Poster Presentation Award in Mathematics and Computer Science

Alex Rollins (2017). Balancing Selection Pressures, Multiple Objectives, and Neural Modularity to Coevolve Cooperative Agent Behavior, Annual Meeting of the Texas Academy of Sciences. (movies, software).

Lauren Gillespie (2016). Picbreeder: The Evolution of Art Inside of a Computer, Southwestern University Science Research Open House. (movies, software).

Gabriela Gonzalez, and Lauren Gillespie (2016). Evolution of Artificial Intelligence in Tetris Play Using Raw Screen Inputs, Southwestern University Science Research Open House. (movies, software).

Alex Rollins (2016). Selection Pressures in Evolutionary Computation, Southwestern University Science Research Open House. (software).

Miscellaneous

Spring 2021: Computers and Creativity: An article about my former student Anna Krolikowski which references me and our work on Zentangles
Spring 2021: Generating Art: An article about my former student Sarah Friday which references me and our work on Zentangles
Fall 2020: Procedural Content Generation for Games with Generative Adversarial Networks: Presentation for the Games AI Research Group at Queen Mary University of London (video)
Fall 2020: Interactively Evolving Video Game Levels with Generative Adversarial Networks: 403 Lecture for Math and CS Department
Fall 2020: Evolving Mega Man Levels with Generative Adversarial Networks: Virtual SCOPE Open House Website
Fall 2020: Evolving Lode Runner Levels with Generative Adversarial Networks: Virtual SCOPE Open House Website
Summer 2020: Generating Video Game Levels Using AI: SCOPE Research Presentation made by my SCOPE Summer research students to present to other SCOPE students
Summer 2019: Playing and Creating Games With Deep Neural Networks: "Mad Science Monday" presentation made by my SCOPE Summer research students to present to other SCOPE students
Spring 2019: Patience, Grit, and an Open Mind: An article about my former student Lauren Gillespie which references me and our work on both Tetris and AnimationBreeder
Spring 2019: Evolving an Intelligent Ms. Pac-Man Agent Under Partially Observable Conditions, presentation by Will Price at the Southwestern University Undergraduate Research & Creative Works Symposium
Spring 2019: Desirable Behaviors for Companion Bots in First-Person Shooters, presentation by Adina Friedman at the Southwestern University Undergraduate Research & Creative Works Symposium
Fall 2018: Musician Presents Research at International Computer Science Conference: An article about my former student Isabel Tweraser which references me and our work on AnimationBreeder
Fall 2018: The cover of SIGEVOlution Volume 11, Issue 4 features art generated by AnimationBreeder, the interactive evolution system described in a GECCO 2018 paper co-authored with SU students. SIGEVO is the ACM Special Interest Group on Genetic and Evolutionary Computation.
Fall 2018: Evolutionary Computation Applied to Digital Entertainment and the Arts, poster presented at the President's Appreciation Celebration for Southwestern University donors.
Fall 2018: Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network: presented to Southwestern University students as a 107 Lecture.
Fall 2018: SCOPE student Will Price (Squillyprice01) won first place in the Ms. Pac-Man track of the Ms. Pac-Man Vs. Ghost Team Competition for our MsPacManEntry.
Summer 2018: Neuroevolution in Video Games: "Mad Science Monday" presentation made by my SCOPE Summer research students to present to other SCOPE students
Spring 2018: Comparing Direct and Indirect Encodings Using Both Raw and Hand-Designed Features in Tetris, presentation by Lauren Gillespie at the Southwestern University Undergraduate Research & Creative Works Symposium
Spring 2018: Querying Across Time to Interactively Evolve Animations, presentation by Isabel Tweraser at the Southwestern University Undergraduate Research & Creative Works Symposium
Fall 2017: Creating Art Through Function Composition: presented to Southwestern University students as a 107 Lecture.
Summer 2017: Evolutionary Computation for Creativity and Intelligence: "Mad Science Monday" presentation made by my SCOPE Summer research students to present to other SCOPE students
Summer 2016: Evolutionary Computation for Creativity and Intelligence: "Mad Science Monday" presentation made by my SCOPE Summer research students to present to other SCOPE students

Last Updated: 9/22/2020