Warning: this project was made some time ago, before I knew how to do Python properly. For example, the code snippets below inexplicably use camelCase. I have left the code unadulterated out of respect for my former naivity.#

Curious about neural networks? It’s surprisingly easy to code one up yourself. And while most how-to guides will take you through the steps of training a neural network on one of many publicly available datasets, I’ve always wanted to watch them learning in real-time. In order to experience that, I’ve coded up a simulation environment in which neural networks can be trained by the rules of the jungle.

The AI#

This is the easy bit. It turns out that natural selection is so much less efficient than machine learning that, when combined with the relatively low sample sizes and very slow rate of iteration in a “natural” simulation, only the very simplest neural networks get anywhere.

I based my neural network on the code shared in this excellent guide: https://towardsdatascience.com/math-neural-network-from-scratch-in-python-d6da9f29ce65

In our simulations, we won’t be training the network in the conventional sense — which means we can discard the backwards propagation layer that calculates the input error for each node (which is needed for gradient descent). Instead, we will need to add a mutation function. I also added a couple of extra activation layer options, including the ubiquitous ReLu function, and a direct function. We’ll see how they perform later.

import numpy as np
import calc

# Base class
class Layer:
    def __init__(self):
        self.input = None
        self.output = None

    # computes the output Y of a layer for a given input X
    def forward_propagation(self, input):
        raise NotImplementedError
        
        
# Fully Connected Layer
class FCLayer(Layer):
    # input_size = number of input neurons
    # output_size = number of output neurons
    def __init__(self, input_size, output_size):
        self.weights = np.random.rand(input_size, output_size) - 0.5
        self.bias = np.random.rand(1, output_size) - 0.5

    # returns output for a given input
    def forward_propagation(self, input_data):
        self.input = input_data
        self.output = np.dot(self.input, self.weights) + self.bias
        return self.output

    # makes random changes to neuron weights and biases
    def mutate(self, mutation_rate):
        for i, weight in enumerate(self.weights):
            self.weights[i] = calc.mutate(self.weights[i], evolution_rate)
        for i, bias in enumerate(self.bias):
            self.bias[i] = calc.mutate(self.bias[i], evolution_rate)
    
    
# Activation Layer
class ActivationLayer(Layer):
    def __init__(self, activation, activation_prime):
        self.activation = activation
        self.activation_prime = activation_prime

    # returns the activated input
    def forward_propagation(self, input_data):
        self.input = input_data
        self.output = self.activation(self.input)
        return self.output

    def mutate(self, mutation_rate):
        pass

    
# activation function and its derivative
def tanh(x):
    return np.tanh(x)

def tanh_prime(x):
    return 1-np.tanh(x)**2

# derivative is the same for relu
def relu(x):
    return np.maximum(0, x)

# for hidden layer that does not change data
def direct(x):
    return x


class Network:
    def __init__(self):
        self.layers = []
        self.loss = None
        self.loss_prime = None

    # add layer to network
    def add(self, layer):
        self.layers.append(layer)

    def mutate(self, mutation_rate):
        for layer in self.layers:
            layer.mutate(mutation_rate)

    # predict output for given input
    def predict(self, input_data):

        output = input_data

        # forward propagation
        for layer in self.layers:
            output = layer.forward_propagation(output)

        return output[0]

Instead of incrementing neuron parameters in the direction that reduces error, as in machine learning, we simply increment neuron parameters in a random direction — machine evolving, if you will.

Let’s make use of numpy’s random.normal() function, as it returns random numbers on a normal distribution with a standard deviation of your choosing. We want normally distributed randomness so that most of the time, neuron parameters won’t change much — the outliers will be the meaningful mutations. And, being able to control the spread of the distribution gives us an elegant way to control the rate of mutation with a single variable.

def applyMutation(n, mutation_rate):

    n += 0.5
    n *= np.random.normal(loc=1, scale=mutation_rate, size=1)
    n -= 0.5

    return n

The simulation environment#

This is a little trickier. We need to to generate agents, process inputs on their behalf (eg vision), and code up mechanisms to translate the network’s outputs into motion. We also need rules that govern life, death, and reproduction. We don’t know how well things are working unless we can watch how things go — so we’ll need to render the results to a window.

Vision#

I experimented with a few different models for vision. In the first simulation below, agents have several sightlines. Each sightline produces a signal when it hits a wall or an obstacle (the only form of which is a circle). This means that we have to solve for each line’s intersections with each obstacle, which quickly gets computationally expensive. This works really well for avoiding large obstacles, but fails at enemy detection. Other entities are too small to efficiently detect, which means huge numbers of sightlines are necessary. This is not just beyond the scope of what’s reasonably to process — it also results in far too many inputs for our neural networks to feasibly mutate towards effective exploitation.

class Eye:

    def __init__(self, parent, angle=0, length=20):

        self.parent=parent
        self.angle=angle
        self.length=length
        self.obstacleDetected=False

    def detectObstacles(self, obstacles):

        self.rayStart = self.parent.location
        angle = self.parent.angle + self.angle
        self.rayEnd = calc.addVectors(self.rayStart, calc.scaleVector(calc.vectorFromAngle(angle), self.length))

        for obstacle in obstacles:
            if lineIntersectsCircle(self.rayStart, self.rayEnd, obstacle['centre'], obstacle['radius']) > 0:
                self.obstacleDetected = True
                return 1

        # wall detection
        if calc.outOfBounds(self.rayEnd):
            self.obstacleDetected = True
            return 1

        self.obstacleDetected = False
        return 0

    def draw(self):
        colour = "grey" if self.obstacleDetected == False else "red"
        pygame.draw.line(
            constants.screen, 
            colour, 
            [self.rayStart[0], calc.flipY(self.rayStart[1])], 
            [self.rayEnd[0], calc.flipY(self.rayEnd[1])],
            1
        )


def lineIntersectsCircle(lineStart, lineEnd, circleCentre, circleRadius):

    d = subVectors(lineEnd, lineStart)       # direction vector of ray from start to end
    f = subVectors(lineStart, circleCentre)       # vector from centre circle to ray start

    a = np.dot(d, d)
    b = 2 * np.dot(f, d)
    c = np.dot(f, f) - circleRadius*circleRadius

    disc = b*b - 4 * a * c

    hit = 0

    if disc >= 0:

        disc = math.sqrt(disc)
        t1 = (-b - disc) / (2 * a)
        t2 = (-b + disc) / (2 * a)

        if (t1 >= 0) & (t1 <= 1):
            hit = 1

        elif (t2 >= 0) & (t2 <= 1):
            hit = 1

    return hit

The next thing I tried was a personal space model. In this model, agents have an area of awareness defined by a radius. Anything in the personal space will always be detected, which solves the sightline blind spot problem. Detected enemies register as inputs in one or more fields of view. In its simplest form, agents will have two input FoVs / channels — one left, and one right.

Increasing the number of channels should increase the potential intelligence of the agent. But in our natural selection environment, input complexity exponentially decreases the likelihood of an effective combination of neural network parameters. I couldn’t get this to work well.

Going for a less realistic form of vision solves this problem. In the second hunter-prey simulation, agents have three vision input channels — one is activated if an enemy is within detection radius, one is the distance to the nearest enemy, and one is the angle to the nearest enemy (with straight ahead being 0, left being negative, right being positive).

def detectEnemies(self, enemies):

    heading = None

    enemyInRange = 0
    nearestEnemyDistance = 0
    nearestEnemyAngle = 0

    nearestEnemyVector = None

    for enemy in enemies:

        vector = calc.getLineVector(self.parent.location, enemy.location)
        distance = np.linalg.norm(vector)

        if distance < (self.detection_range + enemy.radius):

            enemyInRange = 1
            d = 1/distance
            if d > nearestEnemyDistance:
                nearestEnemyDistance = d
                nearestEnemyVector = vector

    if enemyInRange == 1:

        nearestEnemyAngle = calc.signedTheta(heading, nearestEnemyVector)

    return [enemyInRange, nearestEnemyDistance, nearestEnemyAngle]

Mutation rate#

It quickly becomes apparent that evolution rate has a significant impact on the success of natural selection. Something of a goldilocks effect emerges. Too low, and beneficial mutations are not likely. Too high, and beneficial mutations are quickly erased by further mutations. I brute forced my way towards optimal parameters by running dozens of simulations overnight with a range of different settings.

Unnatural selection#

In nature, “better” genes are selected if/when they lead to increased reproduction. This is inefficient, noisy, and subject to randomness in our limited sample sizes. We also have the option of defining a goal and rewarding progress towards it. In the first simulation below, agents compete to get closest to a goal while avoiding obstacles — the subsequent generation is simply bred from the top x% of performers.

Natural selection#

In order to simulate natural selection, we must define rules of life, death, and reproduction. Hunters survive and reproduce if they can catch enough prey. Prey survive if they are not eaten, and reproduce if they live long enough.