A multi-agent evolutionary robotics framework to train spiking neural networks

This repository accompanies the talk and paper, "Training Spiking Neural Networks with a Multi-Agent Evolutionary Robotics Framework" presented at the 2021 Genetic and Evolutionary Computation Conference (GECCO '21), July 10 -- 14, 2021 at Lille, France.
DOI: 10.1145/3449639.3459329
ISBN: 978-1-4503-8350-9/21/07
Copyright 2021

Authors:
Souvik Das (souvik@purdue.edu), Department of Physics and Astronomy
Anirudh Shankar (shanka19@purdue.edu), School of Industrial Engineering
Vaneet Aggarwal (vaneet@purdue.edu), School of Industrial Engineering
Purdue University, Indiana. USA

This work derives from studies conducted by Souvik Das in November 2013. https://github.com/souvik1982/Brain.

Abstract: A novel multi-agent evolutionary robotics (ER) based framework, inspired by competitive evolutionary environments in nature, is demonstrated for training Spiking Neural Networks (SNN). The weights of a population of SNNs along with morphological parameters of bots they control in the ER environment are treated as phenotypes. Rules of the framework select certain bots and their SNNs for reproduction and others for elimination based on their efficacy in capturing food in a competitive environment. While the bots and their SNNs are given no explicit reward to survive or reproduce via any loss function, these drives emerge implicitly as they evolve to hunt food and survive within these rules. Their efficiency in capturing food as a function of generations exhibit the evolutionary signature of punctuated equilibria. Two evolutionary inheritance algorithms on the phenotypes, Mutation and Crossover with Mutation, are demonstrated. Performances of these algorithms are compared using ensembles of 100 experiments for each algorithm. We find that Crossover with Mutation promotes 40% faster learning in the SNN than mere Mutation with a statistically significant margin.

Dependencies

• The high-energy physics data analysis package, ROOT is used for visualization, animation and statistics.

Building

Compilation and linking sequences are specified in the Makefile. Use "make" to generate the executable "BrainInWorld".

Running

The executable, BrainInWorld, can be run with the set of switches listed below.

• -debug: Sets the debugging level. The binary representation of the number following "-debug" determines the debug level. The number must be less than 16 and therefore consists of 4 bits. Each bit determines whether or not a certain debugging feature is turned on.
  • bit 0: Animation.
  • bit 1: Verbose output. Used to check food eating, reproduction, naming and ancestry of bots.
  • bit 2: Fill histograms for future analysis.
  • bit 3: Draw histograms live.
• -skipGenerations: Number of generations to skip visualization.
• -endGenerations: Number of generations at which to end the simulation.
• -timeStep: Number of time-steps at which to draw to canvas for the animation. Smaller timeStep values will slow down execution but produce more time-detailed animation.
• -worldSize: Size of the square world that the bots inhabit.
• -nBots: Number of bots.
• -nFoods: Number of "food" particles.
• -nPredators: Number of predator bots.
• -seed: Set a random number seed for reproducible but pseudo-random results for ensemble averages.

Instructions

1. To simulate 100 runs of the evolutionary algorithm variants create a folder for each variant.
2. The instructions provided ahead are for the "Mutation only" variant but can also be replicated for other variants.
3. Create a folder for the mutation only variant let us say "evolution_mutation".
4. Navigate into this folder and create 100 more folders for 100 random runs. You may name them as "evolution_mutation_seed_1", "evolution_mutation_seed_2" ... "evolution_mutation_seed_100".
5. Within each folder there should be a "run.sh" file for simulating the expirement pertaining to a single seed.
6. The parameters of the "run.sh" can be changed accordingly and the details pertaining to the parameters is given in the table above.
7. Once a run within a folder is completed, you can generate a plot of timesteps to food versus generations by running the following command within the folder: "root -l -b -q '~/Spiking-Evolution/AnalysisScripts/DisplayPlots.cc("evolution_mutation_seed_'insert seed number here'")'.
8. Repeat the procedure for all the 100 seeds.