Design and Implementation of A Central Pattern Generator (CP

Design and Implementation of a Central
Pattern Generator (CPG) Control System

for a Biped Robot

Prepared by:
Alexander F. Russell
Department of Electrical Engineering
University of Cape Town
South Africa

Supervisor:
Dr Ralph Etienne-Cummings
Department of Electrical and Computer Engineering
Johns Hopkins University
United States of America

This thesis is submitted in partial fulfilment of the requirements for the
degree of Bachelor of Science in Engineering (BSc Eng) in Mechatronics

October 23, 2006

ii
Homunculus: An exceedingly minute body that according to
medical scientists of the 16
th
and 17
th
centuries was
contained in a sex cell and whose preformed structure
formed the basis for the human body.
Stedmans Medical Dictionary

A possible Homunculus controlling walking (figure from Inman et al 1981)

iii
Plagiarism Declaration
I, Alexander F. Russell, declare that the work on which this thesis is based is my own original
work (except where the acknowledgements indicate otherwise) and that neither the whole, nor
any part of it, has been or will be submitted for any other degree at this or any other university.

___________________________________
Signed

___________________________________
Date

iv
ACKNOWLEDGEMENTS
Thank you to:

Dr Ralph Etienne-Cummings for supervising the project.

Samuel Ginsberg for an endless supply of Microchip products and C advice.

Prof. John Greene for help with Genetic Algorithms.

Dr Kit Vaughan for help in obtaining human walking data.

v
SYNOPSIS
In the near future it is envisaged that robots will be a part of our everyday lives. Their
involvement will include helping with everyday chores, caring for the sick or elderly, and
performing hazardous tasks.

Currently, the biggest obstacle robots face in achieving this vision is a reliable way to navigate
complex terrains. Legged robots provide the solution to this navigation problem, however, much
research is still needed before they can become a reality.

By building the control systems for the robots using biological principles, such as Central Pattern
Generators and spiking neurons, the systems may be more easily compatible with other biological
principles and biomedical applications such as neuroprosthesis (Lewis et al, 2005).

The aim of this thesis was to design and implement a software based Central Pattern Generator,
using spiking neurons, on the RedBot biped robot. The scope of the project included the design of
the controlling algorithm as well as the necessary hardware and software to implement it.

Literature review
A Central pattern generator (CPG) is a collection of neurons in the spinal chord that are able to
endogenously (i.e. without external input) produce sustained rhythmic or oscillatory, patterned
outputs (Hooper, 2000). The outputs of central pattern generators are responsible for most
rhythmic motor patterns (Marder and Calabrese, 1996; Stein et al, 1997) such as walking, flying,
swimming and breathing.

Izhikevich (2004) shows that as the biological plausibility of a neuron model increases so does its
computation cost. It follows, that simple neuron models are better suited to CPG design where
there is limited processing power available. Lewis et al (2005) showed that it is possible to use
simple Integrate and Fire with Spike Frequency Adaptation neurons to build a CPG capable of
producing joint angle profiles that allowed the RedBot robot to walk. This network is called the
Canonical network.

vi
One of the problems of using spiking neural networks is that they are not amendable to
mathematical analysis and it is a difficult process to find the parameters for these networks
through trial and error.

Lewis et al (1992) showed that it was possible to solve the parameters for these networks using
Genetic Algorithms (GAs). More recently, Inda and Ishii (2004) used a GA to evolve a CPG
controller for biped walking using recorded human movements as the CPGs target output.

The development and simulation environment
A simulation of the RedBot was programmed in MATLAB. The simulation accepts a vector of
joint angle profiles and displays a simple visualisation of the RedBot walking with them. The
simulation calculates the torques, velocities and accelerations required to implement the walking
pattern.

The hardware
The Microchip dsPIC30F4011 microcontroller was chosen to implement the CPG
algorithms and to interface the RedBot to the host PC.

A simple connection board was built to power the RedBot and to interface its various
signal wires to the dsPIC30F4011.

Adapting the Canonical Network
The Canonical Network proposed by Lewis et al (2005) produced a simplified knee angle profile
to that of a human knee angle profile. The Canonical Network was adapted to produce a more
realistic knee angle profile. The resulting network was called the Double Bump Network. Values
were found, through trial and error, for the Double Bump Network to cause joint angle profiles
for walking at a frequency of 1.5Hz.

The Double Bump Network was implemented on the dsPIC30F4011 and the RedBot walked
successfully, however a small amount of slip was observed between the RedBots feet and the
ground.

vii
The frequency of walking of the RedBot was not at the desired 1.5Hz but instead at 0.75Hz. This
was a result of the dsPIC30F4011 not being able to process the Double Bump Network in real
time.

Automating the search for parameters
An automated method, using a Genetic Algorithm called FBGAMIN, was designed to find the
necessary parameters to cause oscillations of a CPG, over a frequency range of 0.5Hz to 3.5Hz,
with prescribed phase shifts between specific neurons within the CPG.

The principles of Behavioural Memory (de Garis, 1990) and staged evolution (Lewis et al, 1992)
were used to increase the rate of convergence of FBGAMIN.

Parameters were found to cause oscillations at all the desired frequencies and phase shifts were
accurate to within 1. The convergence rate of the search was extremely fast with a maximum of
four generations needed to find any solution.

The parameters found, using this method, were applied to the Canonical Network. Visual
inspection of the resultant joint angle trajectories suggested that the RedBot could walk with
them. However, insufficient time was available to physically test these joint angle trajectories on
the RedBot.

Evolution of a CPG controller
Human joint angles, recorded during normal walking, were used as target joint angle profiles for
a CPG whose parameters were evolved using FBGAMIN. The resultant joint angle profiles
closely approximated the human joint angle profiles. FBGAMIN was then used to make the CPG
as efficient as possible by removing any redundant neurons within the network.

The evolved network was too large to be simulated on the dsPIC30F4011 and thus a single
periods worth of walking patterns was simulated in MATLAB and then uploaded to the
microcontroller for testing on the RedBot.

The RedBot walked extremely well using the evolved CPG, however a small amount of slip
between the RedBots feet and the ground was observed.

viii
Conclusions
Two Central Pattern Generators, using different network architectures, were designed and both
caused successful walking of the RedBot robot. However, the algorithms proved too complex to
be implemented on the dsPIC30F4011 microcontroller in real time. Both the CPGs designed
outputted more physiologically accurate joint angle profiles than Lewis et als Canonical
Network, with the joint angle profiles generated by the evolved controller being very accurate to
the joint angle profiles of a walking human.

In addition to this, an automated search method was developed to find parameters for the
Canonical Network to generate joint angle profiles for the RedBot at any frequency between
0.5Hz and 3Hz.

The following paper was a direct result of this work and has been submitted to the International
Symposium on Circuits and Systems(ISCAS) 2007 for publishing.
Russell A, Orchard G and Etienne-Cummings R, Configuring of Spiking Central Pattern
Generator Networks for Bipedal Walking Using Genetic Algorithms.

Recommendations
The methods used to implement the CPGs on the microcontroller should be changed so
that the CPGs can run in real time. This could be achieved in many ways, however, the
first thing to be redesigned should be the filter.

The feet of the RedBot should be altered to provide better traction.

The method used in the automated parameter search should be applied to hardware CPG
networks instead of software networks. This would allow the GA to optimise the network
in real time (possibly even faster) allowing for on-line reconfiguration of the network
parameters to adjust for any desired change in gait.

ix
TABLE OF CONTENTS
PLAGIARISM DECLARATION III
ACKNOWLEDGEMENTS IV
SYNOPSIS V
TABLE OF CONTENTS IX
LIST OF FIGURES XIV
LIST OF TABLES XIX
1. INTRODUCTION 1
1.1. BACKGROUND TO STUDY 1
1.2. PROJECT DESCRIPTION 2
1.2.1. OBJECTIVES OF THIS THESIS 2
1.2.2. WORK BREAKDOWN 2
1.3. THESIS OUTLINE 3
1.4. LIMITATIONS 5
1.5. INTRODUCTION TO EQUIPMENT 5
1.6. CONVENTIONS USED 6
1.7. GLOSSARY 7
1.7.1. TERMS RELATED TO GAIT 7
1.7.2. TERMS RELATED TO NEURONS 8
1.8. ACRONYMS 8
2. BACKGROUND TO WALKING ROBOT CONTROL 9
2.1. WALKING ROBOT CLASSIFICATION METHODS 9
2.2. METHODS OF CONTROLLING WALKING ROBOTS 10

x
2.2.1. TRAJECTORY BASED METHODS 10
2.2.2. RULE BASED METHODS 10
2.2.3. BIOLOGICALLY INSPIRED METHODS 10
3. CENTRAL PATTERN GENERATOR LITERATURE REVIEW 11
3.1. INTRODUCTION AND SCOPE OF REVIEW 11
3.2. WHAT IS A CENTRAL PATTERN GENERATOR? 11
3.3. A BRIEF HISTORY OF CENTRAL PATTERN GENERATORS 11
3.4. SUCCESSFUL IMPLEMENTATIONS OF CPGS IN LEGGED ROBOTS 12
3.5. CONSTRUCTION OF CENTRAL PATTERN GENERATORS USING NEURONS 13
3.5.1. SPIKING NEURON MODELS AND THEIR AFFECT ON CPG DESIGN 13
3.5.2. CPG DESIGN USING SPIKING NEURONS 14
3.5.3. CPG DESIGN USING NON-SPIKING NEURON MODELS 16
3.5.4. PROBLEMS WITH CPG DESIGNS USING BI-STABLE OSCILLATORS 16
3.5.5. ALTERNATIVE APPROACH TO CPG DESIGN 16
3.6. ENDOGENOUS OSCILLATOR NEURON APPROACH 16
3.7. CONSTRUCTION OF CPGS USING NON-NEURAL OSCILLATORS 17
3.8. CONCLUSIONS 17
4. BACKGROUND TO GENETIC ALGORITHMS 18
4.1. WHAT IS A GENETIC ALGORITHM? 18
4.2. INCREASING THE RATE OF CONVERGENCE 19
4.3. TYPES OF GENETIC ALGORITHMS 19
4.3.1. NSGA-II 19
4.3.2. FBGAMIN 19
4.4. APPLICATIONS OF GENETIC ALGORITHMS TO CPG NETWORKS 19
4.5. CONCLUSIONS 20
5. THE DEVELOPMENT AND SIMULATION ENVIRONMENT 21
5.1. CONVENTIONS AND ASSUMPTIONS 21
5.2. THE SIMULATION PROCESS AND CALCULATIONS 23

xi
5.3. THE FINAL DEVELOPMENT GUI 25
6. HARDWARE 27
6.1. INTRODUCTION TO THE REDBOT 27
6.2. THE MICROCONTROLLER 28
6.3. REDBOT TO DSPICDEM2 CONNECTION BOARD 29
7. PRELIMINARY METHODS 30
7.1. EXTRACTING HUMAN WALKING DATA FROM GAITLAB 30
7.2. 4
TH
ORDER RUNGE-KUTTA METHOD 30
7.3. SIMULATION OF A SINGLE NEURON 31
8. PRELIMANRY RESULTS 35
8.1. EXTRACTION OF HUMAN WALKING DATA 35
8.2. 4
TH
ORDER RUNGE-KUTTA RESULTS 36
8.3. SIMULATION OF A SINGLE NEURON RESULTS 37
9. TRIAL AND ERROR CPG METHODS 39
9.1. CONSTRUCTION OF A HALF-CENTRED OSCILLATOR 39
9.2. REFINING THE NEURON MODEL 40
9.3. USING GENETIC ALGORITHMS TO AID TRIAL AND ERROR 43
10. TRIAL AND ERROR CPG RESULTS 45
10.1. HALF-CENTRED OSCILLATOR RESULTS 45
10.2. REFINING THE NEURON MODEL RESULTS 46
10.3. USING GENETIC ALGORITHMS TO AID TRIAL AND ERROR RESULTS 48
11. FILTER AND SCALING DESIGN METHOD 50
12. FILTER AND SCALING RESULTS 52

xii
13. DIFFERENT CPG NETWORK ARCHITECTURES 54
13.1. CANONICAL NETWORK 55
13.2. DOUBLE BUMP NETWORK 56
13.3. MODIFICATIONS OF NETWORKS TO REDUCE COUPLING WEIGHTS 58
14. C IMPLEMENTATION 61
14.1. BASIC CODE STRUCTURE 61
14.2. REDUCING THE CPG NETWORKS 62
14.3. IMPLEMENTATION OF THE DOUBLE BUMP NETWORK ON THE REDBOT 63
15. EVOLUTION OF CPG PARAMETERS METHOD 66
15.1. AUTOMATING THE SEARCH FOR PARAMETERS 66
15.1.1. STAGE 1: EVOLVING OSCILLATOR PARAMETERS 68
15.1.2. STAGE 2: EVOLVING PHASE SHIFT PARAMETERS 72
15.2. EVOLUTION OF A CONTROLLER TO SIMULATE HUMAN WALKING 77
16. EVOLUTION OF CPG PARAMETER RESULTS 80
16.1. AUTOMATING THE PARAMETER SEARCH: STAGE 1 RESULTS 80
16.2. AUTOMATING THE PARAMETER SEARCH: STAGE 2 RESULTS 82
16.2.1. RESULTS OF A SINGLE PHASE SHIFT AT VARIABLE FREQUENCIES 82
16.2.2. RESULTS OF MULTIPLE PHASE SHIFTS AT A SINGLE FREQUENCY 83
16.2.3 DISCUSSION OF RESULTS 84
16.3. APPLICATION OF GA RESULTS TO CANONICAL NETWORK 84
16.4. EVOLUTION OF A HUMAN CONTROLLER RESULT 86
16.5. IMPLEMENTATION OF THE EVOLVED CPG ON THE REDBOT 90
17. CONCLUSIONS AND RECOMMENDATIONS 93
17.1. CONCLUSIONS 93
17.2. RECOMMENDATIONS 94

xiii
18. REFERENCES 96
19. BIBLIOGRAPHY 100
20. APPENDICES 101
20.1. APPENDIX 1 101
20.6. APPENDIX 6 (SEE ATTACHED CD) 109

xiv
LIST OF FIGURES
Figure 1 The different planes of the human body
Figure 2 Comparison of Neuron Model Implementation Cost and Biological Plausibility
Figure 3 A half-centred oscillator
Figure 4 Canonical network design (Lewis et al, 2005)
Figure 5 Naming conventions for RedBot simulation
Figure 6 RedBot weights and lengths used in the simulation
Figure 7 Torque calculation flow diagram
Figure 8 The development and simulation GUI
Figure 9 The RedBot
Figure 10 Algorithm for CPGFRAMEWORK1
Figure 11 Algorithm for the IFA function
Figure 12 Hip and Knee Angle Trajectories during normal human walking
Figure 13 Runge-Kutta results vs. actual solution
Figure 14 Results of Neuron Simulation experiment to illustrate tonic spiking
Figure 15 Results of Neuron Simulation Experiment Showing the Neurons Spike
Frequency Adaptation Characteristic
Figure 16 the half-centred oscillator layout
Figure 18 Membrane Potentials of a half-centred oscillator with an initial frequency of
0.5Hz
Figure 19 Firing patterns of a half-centred oscillator with a starting frequency of 0.5Hz
Figure 20 Potassium concentrations of a half-centred oscillator with a starting frequency
of 0.5Hz
Figure 21 Membrane potential of a half-centred oscillator, using IF&A2 neurons, at
0.5Hz
Figure 22 Firing patterns of a half-centred oscillator, using IF&A2 neurons, at 0.5Hz
Figure 23 Potassium concentrations of a half-centred oscillator, using IFA2 neurons, at
0.5Hz
Figure 24 Results of NSGA-II first trial

xv
Figure 25 Results of NSGA-II second trial
Figure 26 Results of NSGA-II third trial
Figure 27 Results of NSGA-II fourth trial
Figure 28 Half-centred oscillator at 1.5Hz using parameters found through NSGAII and
trial and error
Figure 29 algorithm for LPF (Low Pass Filter)
Figure 30 Input spike train to filter
Figure 31 Power Spectral Density of input signal
Figure 32 scaled filtered signal with fc
1
=15Hz, fc
2
=1Hz, A = 0 and B = 0
Figure 33 Enlarged view of ripple and sampling on scaled signal
1
=15Hz, fc
2
=1Hz, A = 140 and B =15
Figure 35 Canonical Network with summing nodes and filtering
Figure 36 Joint angle trajectories from Canonical network
Figure 37 Double Bump Network with summing nodes and filtering
Figure 38 Joint Angle profiles from the Double Bump Network
Figure 39 Canonical Network 2
Figure 40 Double Bump Network 2
Figure 41 Joint angle profiles generated from Double Bump Network 2
Figure 42 MAIN algorithm
Figure 43 reduced CPG network architecture
Figure 44 RedBot walking at 0.75Hz (1.5Hz desired) using Double Bump 2 Network
Figure 45 Joint angle profiles obtained whilst the RedBot was walking
Figure 46 CPG network layout used in the automated search for parameters
Figure 47 Algorithm for automated parameter search to cause oscillations of CPG1 and
CPG2 at different frequencies
Figure 48 Overview of OSCILLATIONS
Figure 49 Detailed algorithm of OSCILLATIONS
Figure 50 Algorithm for automated search for parameters to cause a single phase shift
between CPG1 and CPG3 at different frequencies
Figure 51 Algorithm for automation of parameter search to cause different phase shifts
between CPG1 and CPG3 at a single frequency.

xvi
Figure 52 Overview of PHASES
Figure 53 A detailed algorithm of PHASES
Figure 54 Initial neural network layout used to evolve both the hip and knee controllers
Figure 55 Algorithm to evolve human hip or knee controller
Figure 56 Actual frequencies of oscillation vs. desired frequencies of oscillation using
parameters found by FBGAMIN
Figure 57 Absolute values of parameters found by FBGAMIN vs. the frequency of
oscillation they cause
Figure 58 Number of generations FBGAMIN required to find the parameters to cause
oscillations at each frequency
Figure 59 Actual phase shift vs. desired 90 phase shift at different frequencies using
Figure 60 Actual phase shift vs. desired 45 phase shift at different frequencies using
Figure 61 Absolute values of parameters found by FBGAMIN to cause a phase shift of
90 at different frequencies of oscillation
Figure 62 Absolute values of parameters found by FBGAMIN to cause a phase shift of
45 at different frequencies of oscillation
Figure 63 Number of generations FBGAMIN required to find the parameters to cause a
phase shift of 90 at each frequency.
Figure 64 Number of generations FBGAMIN needed to find the parameters to cause a
phase shift of 45 at each frequency.
Figure 65 Actual phase shifts vs. desired phase shifts at oscillations of 1Hz using
Figure 67 Absolute value of parameters found using FBGAMIN vs. the phase shifts they
cause at 1Hz
Figure 68 Absolute value of parameters found using FBGAMIN vs. the phase shifts they
cause at 2Hz
particular phase shift at 1Hz

xvii
Figure 71 Hip and knee angles for 0.5Hz walking
Figure 72 Hip and knee angle for 1Hz walking
Figure 74 Hip and knee angles for 2Hz walking
Figure 76 Hip and knee angles for 3Hz walking
Figure 77 Neuron network to simulate human hip trajectory
Figure 78 Neuron network to simulate human knee trajectory
Figure 79 Hip simulation results
Figure 80 Knee simulation results
Figure 81 Spike trains of the surviving neurons for the hip controller
Figure 82 The weighted sum of CPG3s, CPG5s and CPG8s outputs
Figure 83 Spike trains of surviving neurons for the knee controller
Figure 84 The weighted sum of CPG3s, CPG6s, CPG7s and CPG8s outputs
Figure 85 RedBot walking using evolved controller
Figure 86 Recorded hip and knee joint angle profiles during walking
Figure 87 Actual phase shift vs. desired 90phase shift at different frequencies using
parameters found by FBGAMIN, with I
2-3
scaled by 1000
Figure 88 Number of generations FBGAMIN needed to find parameters causing a phase
shift of 90 at each frequency, where I
2-3
was scaled by 1000
Figure 89 Actual phase shift vs. desired 45phase shift at different frequencies using
parameters found by FBGAMIN, with I
2-3
scaled by 1000
Figure 90 Number of generations FBGAMIN needed to find parameters causing a phase
shift of 45 at each frequency, where I
2-3
was scaled by 1000
parameters found by FBGAMIN, where I
2-3
was scaled by 1000
particular phase shift at 1Hz, where I
2-3
was scaled by 1000
parameters found by FBGAMIN, where I
2-3
was scaled by 1000

xviii
Figure 94 Number of generations FBGAMIN required to find parameters to cause a
particular phase shift at 2Hz, where I
2-3
was scaled by 1000

xix
LIST OF TABLES
Table 1 Joint Servo Motor Names and Characteristics
Table 2 Requirements for microcontroller used to control the RedBot
Table 3 dsPIC30F4011 pin layouts
Table 4 Parameters to test the IFA2 neuron model
Table 5 Maximum and minimum parameter values for nsga_2 Function
Table 6 Parameters to cause oscillations of a half-centred oscillator at a steady state
frequency of 0.63Hz
Table 7 parameters found using NSGAII for oscillations at 1.5Hz
Table 8 parameters for Canonical network to generate walking patterns at 1.5Hz
Table 9 parameters to cause 1.5Hz oscillations and 90 shifts in Canonical Network 2 and
Double Bump Network 2
Table 10 Trial solution for 0.5Hz oscillations
Table 11 showing the neuron parameters to illustrate tonic spiking and spike frequency
adaptation
Table 12 parameters to cause oscillations of CPG1 and CPG2 at different frequencies
Table 13 parameters to cause 45 and 90 phase shifts between CPG1 and CPG3 at
frequencies ranging from 0.5Hz to 3.5Hz
Table 14 parameters to cause phase shifts of 90 and 45 for different frequencies of
oscillation
Table 15 Filter parameters used in applying the parameters found by FBGAMIN to Lewis
et als Canonical Network
Table 16 Trial and final solutions for the evolution of hip and knee controllers

1
1. INTRODUCTION
1.1. Background to study
In the near future it is envisaged that robots will be a part of our everyday lives. Their
involvement will include helping with everyday chores, caring for the sick or elderly, and
performing hazardous tasks.

Currently, robots, such as autonomous vacuum cleaners, already exist to perform some of these
functions. However, these robots are generally wheel based and are thus limited in the type of
terrain they can navigate. Walking robots, being able to traverse varying terrains, would be far
more effective in performing these tasks. However, walking robots are much more complicated
than wheeled ones and much research is required before they can be produced for mass use.

The research around bipedal walking also has application in the biomedical field. Spinal cord
injuries are often debilitating as lesions to the spinal cord disrupt communications between the
brain and the central and peripheral nervous systems below the lesion, leaving the affected
individual paralysed. This paralysis may be partial paralysis of a limb or in the worst case all
motor control below the lesion may be destroyed. The damaged nerve cells do not grow back,
disabling the affected individual for life. Stem cell research shows promising results in repairing
these damaged cells but the research is a long way away from being feasible. In addition to this,
ethical issues with this research have resulted in it being illegal in many countries.

Mushahwar et al (2000) showed that by microstimulating the spinal cord of cats it is possible to
activate certain muscles in the cats legs, and that if the microstimulation was applied to specific
areas, coordinated limb movements could be generated. With further research, it may be possible
to activate the CPGs of humans using microstimulation, or to use an artificial CPG to activate the
necessary muscles and thus artificially control human walking. For this to happen it is crucial to
have a firm understanding of how Central Pattern Generators work, and to have thoroughly tested
and researched them on robots, before testing is done on humans.

Furthermore, by building the CPG using spiking neurons the CPG may be more easily interfaced
with biological networks, which also operate using spiking neurons.

2
1.2. Project description
This project was to be completed by a team of two. The aim of the project was to implement two
different walking algorithms (a Central Pattern Generator algorithm and an If-Then Rules
algorithm) on the RedBot biped robot. The Central Pattern Generator algorithm was to use
spiking neurons as its building blocks. Each member of the team was to be responsible for
implementing a different algorithm.

The implementation process was to include the design of the walking algorithms, simulating the
RedBot in MATLAB, and building the necessary hardware to interface the RedBot to the host PC
and to control the RedBots servo motors.

The following specifications were given for the project
Control of the RedBot must be achieved using a PIC microcontroller to interface the PC
to the RedBot.
The final deliverables were to be a written report and evidence (e.g. video) of the RedBot
walking using the designed controller.
Work on this thesis was to start on 31 July 2006 and the final report and any evidence of
the RedBot walking were to be submitted on 23 October 2006.
1.2.1. Objectives of this thesis
The author chose to implement the Central Pattern Generator based algorithm and thus the
objective of this thesis was to build a software implemented Central Pattern Generator, using
spiking neurons to control the walking of the RedBot biped robot.
1.2.2. Work breakdown
As this was a joint thesis, certain sections of it were conducted together with Garrick Orchard, the
authors thesis partner. These sections included:
The initial research into the two different walking algorithms
Sourcing the necessary hardware components
Simulating the RedBot in MATLAB

Garrick Orchard was responsible for setting up the skeleton code for the PIC microcontroller and
the author was responsible for building the connection board to supply the RedBot with power

3
and to interface the RedBot to the PIC microcontroller. Garrick orchard was, however,
responsible for adding the necessary hardware to the connection board to allow for foot and shin
switches to be installed on the RedBot.
1.3. Thesis outline
Chapter 2 provides an introduction to walking robot control including different methods of
classifying walking robots and the methods used to control them.

Chapter 3 is the Central Pattern Generator literature review. It provides a brief history of Central
Pattern Generators as well as providing insights into the different methods of implementing
CPGs. The focus of this chapter, however, is on central pattern generators using spiking neurons.

Chapter 4 is the Genetic Algorithm literature review. It provides a very brief introduction into
how Genetic Algorithms work and the terminology associated with them. Examples of previous
applications of Genetic Algorithms to CPG design have been included.

Chapter 5 gives the details of the simulation of the RedBot in MATLAB.

Chapter 6 provides a short description of the different hardware elements used in this thesis.

Chapter 7 describes the methods of the preliminary tasks that had to be performed before the
design of the Central Pattern Generator could commence. This includes giving a detailed
algorithm for the MATLAB code used to simulate a network of n neurons.

Chapter 8 describes the results of the preliminary methods.

Chapter 9 describes how trial and error can be applied to find parameters for a CPG. It also
explains how the Integrate and Fire with Spike Frequency Adaptation neuron model was adapted
to ease the search for parameters.

Chapter 10 gives and discusses the results of a trial and error approach to the design of a Central
Pattern Generator.

4
Chapter 11 shows the details of the digital filter used to filter the resultant spike trains from the
Central Pattern Generator and how the filtered signals are scaled to lie between the desired
angles. It discusses the method used to find filter parameters for different spike trains.

Chapter 12 gives results of the filtering and scaling of a specific spike train and shows how the
method discussed in Chapter 11 is used to find parameters.

Chapter 13 describes different CPG architectures. The CPG network proposed by Lewis et al
(2005) is first discussed and then adaptations of this network, designed by the author, are given.
The joint angle profiles that result from these different networks are shown.

Chapter 14 gives a brief overview of how the CPG code was implemented on the dsPIC30F4011
microcontroller. It also shows the results of implementing the Double-Bump 2 Network
(discussed in Chapter 13) on the RedBot.

Chapter 15 provides a method, to automate the search for parameters, to cause different
frequencies of oscillation of a CPG and phase shifts between the neurons within the CPG. This
chapter also provides the method followed to evolve a CPG controller that mimics human hip and
knee joint angle profiles.

Chapter 16 gives and discusses the results for the automated parameter search. It shows the
results of applying these parameters to Lewis et als Canonical network. The results of the
evolution of a controller are presented and the results of implementing this controller on the
RedBot are shown and discussed.

Chapter 17 This chapter gives conclusions on the thesis and provides recommendations for
further research.

5
1.4. Limitations
The following limitations were encountered during this thesis:
Only 12 weeks were allowed for this project and hence time played a critical factor.
The computers available for this thesis only had 256MB of RAM. This prevented the
Central Pattern Generator algorithms from being run on the computers in real time.
Only versions 19 or higher of the MPLAB ICD2 (The debugger/programmer module for
the PIC) are compatible with the dsPIC30F series of microcontroller. This is not
documented in the Microchip documentation. Initially an incompatible version of the
ICD2 was being used with the dsPIC30F4011 microcontroller, however, as there was no
documentation explaining this incompatibility problem, it was a time consuming process
to discover it.
The ICD2 is not robust and the Microchip documentation regarding the order in which
power is to be supplied to the ICD2 and target PIC is incorrect. The documentation states
that power should first be applied to the ICD2 before it is applied to the target PIC,
however, this permanently damages the ICD2. Power should actually be applied to the
microcontroller first. As a result of the error in the documentation two ICD2s were
damaged and as there are only a limited number of ICD2s available for use at UCT,
delays were encountered whilst waiting for new ICD2s.
Replacement servo motors for the actuation of the RedBot joints are not locally available
in Cape Town. One of the gears in the Futaba s4092 servo motor used to control the
RedBots right hip broke. The RedBot was unusable until a new servo motor could be
sourced from the United States of America.
1.5. Introduction to equipment
The following hardware was used in this thesis:
A 2.4Ghz Pentium 4 with 256MB RAM PC was used for all simulations and testing of
algorithms
A 1.7Ghz Celeron with 256MB RAM PC was used to program and interface with the
dsPIC30F4011 microcontroller
A MPLAB ICD2 programmer and debugger module
A dsPIC30F4011 microcontroller

6
A dsPICDEM2 development board was used. It contains all the peripheral circuitry
needed by the dsPIC30F4011 microcontroller to run and to interface with the host PC.
A RedBot to dsPICDEM2 connection board
A RedBot biped robot

The RedBot, dsPIC30F4011 and connection board are discussed in more depth in Chapter 6.

The following software was used in this thesis
MATLAB 7 Release 14 was used to develop the walking algorithms, simulate the
RedBot and communicate with the PIC microcontroller.
MPLAB IDE 7.41 is the development environment for the PIC microcontroller and
provides support for the MPLAB ICD2 debugger/programmer. All C code was written in
this environment.
MPLAB C30 Compiler (Student edition) is the C compiler for the dsPIC range of
Microchip products.
Microsoft Windows Hyper Terminal was used to display data sent from the PIC to the
PC.
Gaitlab is gait analysis software designed for use on a home PC. Gaitlab provides
examples and analysis of different gaits, where each gait was captured using video-based
kinematics methods. Gaitlab allows the user to extract information such as the joint angle
profiles from this captured data.
1.6. Conventions used
The following conventions have been used:
Names of functions written for use in MATLAB or C are in uppercase italics.
Variable names are in italics.
The term spiking is analogous to a neuron firing. E.g. if a neuron spikes at 1s intervals
then the neuron fires at 1s intervals
Extension of the hip was defined to be a movement of the hip to move the thigh towards
the front of the RedBot
Flexion of the hip was defined to be a movement of the hip to move the thigh towards the
back of the RedBot.
Extension of the knee was defined to be a straightening movement of the knee.

7
Flexion of the knee was defined to be a bending movement of the knee.
In the figures describing the neural network architectures, there is at least a 180 phase
difference between the firing patterns of the white coloured neurons and the gold
coloured neurons. The blue coloured neurons denote that the amount of phase shift of the
blue neurons firing pattern from a white or gold neurons firing pattern is unknown. If
the neuron is green then the neuron fires for more than 50% of the oscillatory period.

It must be noted that while attempts were made to make the Central Pattern Generator
biologically compatible through the use of spiking neurons, no attempt was made to make the
system biologically plausible. The focus was instead on what the resulting neuron spike trains
looked like and whether or not they were useful for generating joint angle profiles for walking.

As a result of this, no restrictions were placed on the parameters to cause the neuron membrane
potentials to lie within any particular range and no scale was set for the values of the neuron
parameters (be it a voltage, current or impedance). E.g. if a parameter was a voltage it was never
specified whether the units of this voltage were megavolts, volts or millivolts. Likewise, for those
parameters that are currents, it was never specified whether the units of current were mega-amps,
amps or milliamps. Consequently it does not make sense to put units on the graphs and tables
showing these parameters as the units are meaningless. However, when the neuron model that is
used is discussed, the parameter types (e.g. voltage, current or impedance) are specified.

It was, however, decided that the spike trains that emerge from the neurons should have the units
of volts, and that a value of 1v would represent the neuron firing and a value of 0v would
represent the neuron not firing.
1.7. Glossary
1.7.1. Terms related to gait
The foot that is in contact with the ground is called the stance foot.
The leg whose foot is the stance foot is called the stance leg.
The foot that is not in contact with the ground is called the swing foot.
The leg whose foot is the swing foot is called the swing leg.

8

Figure 1 The different planes of the human body
(Inman et al, 1981)

1.7.2. Terms related to neurons
The input current to a neuron is called the tonic input.
When a neuron has an applied tonic input it will fire a train of spikes. This behaviour is
known as tonic spiking.
The firing rate of tonic spiking of certain neurons decreases over time. This is known as
spike frequency adaptation
1.8. Acronyms
ADC - Analogue to Digital Converter
COG - Centre of Gravity
COM - Centre of Mass
CPG - Central Pattern Generator
DSP - Digital Signal Processing
GA Genetic Algorithm
IO - Input Output
PIC - Generic name for microcontrollers manufactured by Microchip
PWM Pulse Width Modulation
UART - Universal Asynchronous Receiver Transmitter
UCT University of Cape Town
ZMP - Zero Moment Point

Transverse plane
Sagittal plane
Frontal plane

9
2. BACKGROUND TO WALKING ROBOT
CONTROL
2.1. Walking robot classification methods
There are many methods to classify walking robots, the most obvious being by the number of legs
the robot has. Other methods include classifying according to the type of joint actuation used or
according to the position of the Centre of Mass (COM) whilst walking.

There are three types of actuated robots:
Passive walkers have no joint actuation and rely on gravitational forces to drive the
robot. As a result they are only able to walk down inclined surfaces. These robots are
very efficient as they require no input power to cause walking. Mcgeer (1990) showed
that it was possible to realise stable walking motion through mechanical design only (i.e.
without control input).
Ballistic walkers use joint actuation only during the stance phase of the walking cycle.
The walker uses gravity and inertia to provide the motion of the leg in the swing phase.
Powered walkers use actuation during both the swing and stance phases.

Classification according to the robots position of the Centre of Mass (COM) results in two
classes:
Static walkers ensure that their COM is within the support area of the walkers feet at all
times. As a result they are stable at any position of their feet and at all points during their
walking cycle. However, in order to achieve this stability, the speed and step size of the
walker is limited.
Dynamic walkers use velocity or inertia to achieve stability. The COM is not
constrained to lie within the support are of the feet and as a result the walker is able to
move at higher speeds and with larger step sizes.

The robot used in this thesis falls into the powered, dynamic walker class of robots.

10
2.2. Methods of controlling walking robots
2.2.1. Trajectory based methods
These methods use mathematical equations to calculate the position of each limb and the angle of
each joint of the robot at every moment. There are two main methods based on trajectory:
Zero Moment Point (ZMP) methods specify the movement of the robot such that the
sum of the moments from the active forces sum to zero at a particular point on the ground
(the Zero Moment Point). If the ZMP falls within the area generated by all possible
contact points between the feet and the ground then it is possible for a biped robot to
walk (Huang et al, 2001). Kanniah et al (2004) used the ZMP method to achieve stable
walking by calculating the compensatory motion required by the hips for a given swing
leg trajectory. Kajita et al (2003) specify the placement of the feet and trajectory of the
COM such that the ZMP always coincides with the stance foot.
Inverse Kinematics involves the computation of the required joint angles from pre-
defined joint trajectories
2.2.2. Rule based methods
Rule base methods use predefined rules to determine the next action to be taken by the robot,
given its current state (obtained through sensory feedback) and objective. The rule based method
for walking is commonly called the If-Then rules approach.

In some circumstances it is hard to define an exact line between the different states and in this
case fuzzy logic can be used to determine which rule should be applied (Prochazka, 1996)
2.2.3. Biologically inspired methods
These methods are based on biological principles such as the Central Pattern Generator (CPG).
By basing the control of movement on biological principles it may make the robot more
compatible with other biologically inspired aspects of intelligence (Lewis et al, 2005). Central
pattern generators are discussed in depth in the next chapter.

11
3. CENTRAL PATTERN GENERATOR
LITERATURE REVIEW
3.1. Introduction and scope of review
Central Pattern Generators have successfully been used in controlling a number of different
robots. These CPGs have been implemented either through software or hardware. Both
implementations share many characteristics and thus, although this project involves a software
implemented CPG, papers concerning both implementations have been researched.
3.2. What is a Central Pattern Generator?
A Central pattern generator (CPG) is a collection of neurons in the spinal chord that are able to
endogenously (i.e. without external input) produce sustained rhythmic or oscillatory, patterned
outputs (Hooper, 2000). The outputs of central pattern generators are responsible for most
rhythmic motor patterns (Marder and Calabrese, 1996; Stein et al, 1997) such as walking, flying,
swimming and breathing.

Central Pattern Generators are more than just rhythm control centres responsible for creating
rhythmic, patterned motor control signals. Central Pattern Generators help coordinate the brain
and the spinal chord by providing phase information to the cerebellum (Cohen and Boothe, 1999).
They are also involved in the modulation of sensory information from the periphery (Arshavsky
et al, 1983), as well as providing the additional state variable over sensory information which is
necessary to understand occurrences such as reflex reversal (Forssberg et al, 1975).
3.3. A brief history of Central Pattern Generators
Towards the end of the 19
th
century scientists started studying the neural mechanics responsible
for locomotion and the first evidence of a central pattern generator was discovered at the
beginning of the 20
th
century by T. G. Brown. He was experimenting with dogs and discovered
that locomotor-like patterns persisted after sensory input was destroyed by extensive surgical
transection of sensory roots entering the spinal chord. Brown concluded the reason for this was an
intrinsic factor within the spinal cord, which autonomously produced the rhythm and that the
role of sensory input was to control this intrinsic factor to manage variations and abnormalities
in the external environment. After 1920 there was a lull in the research of complex movement

12
control; however a renewed interest in this field occurred in the 1960s and the intrinsic factor
described by Brown was named the central pattern generator or CPG (Prochazka, 2002).

Wilson (1961) demonstrated that when the locust nervous system is removed from the animal, it
is able to produce a rhythmic output similar to that observed during flight, and thus provided the
first modern evidence that rhythmic motor patterns are centrally generated. Additional research
has shown similar results in a wide variety of animals, further demonstrating that rhythmic
pattern generation is not dependant on the nervous system acting as a whole, but rather on small
autonomous systems of neural networks (Hooper, 2000). Various studies have provided evidence
of the existence of a central pattern generator in humans (Bussel et al, 1996; Duysens, Van der
Crommert, 1998; Dimitijevic et al, 1998).
3.4. Successful implementations of CPGs in legged robots
Central Pattern Generators have successfully been implemented, in both hardware and software,
to control a variety of legged robots. Inagaki et al. (2006) used a software implemented wave
Central Pattern Generator model to provide decentralized control of gait generation and walking
speed of a hexapod robot. Hata et al (2006) used a pulse-type hardware model to successfully
control a quadruped robot, achieving the following gaits: walk, pace, trot, bound and gallop. Pinto
and Golubitsky (2006) used a software implementation of a Central Pattern Generator to achieve
the five gaits described above, as well as eight additional gaits. Pinto and Golubitsky (2006) also
used the software implementation of a Central Pattern Generator for biped gait control. They
achieved the following gaits: walk, run, two-legged hop, two-legged jump, skip, gallop,
asymmetric hop and one-legged hop. Tenore et al (2005) used a hardware implemented Central
Pattern Generator to control the RedBot biped robot.

The above references are not meant to be a listing of the only successful implementations of
Central Pattern Generators in robot control, instead they are meant to show the variety of legged
robots they have been controlled using central pattern generators.

13
3.5. Construction of Central Pattern Generators using neurons
3.5.1. Spiking neuron models and their affect on CPG design
There are many different spiking neuron models available ranging from the extremely accurate
and biologically plausible Hodgkin-Huxley model to the very basic leaky integrate and fire (IF)
model. Izhikevich (2004) provides a detailed discussion of the different modes of spiking neurons
and discusses the neuron models available. He shows that with an increase in biological accuracy
comes an increase in the computational time needed to process the neuron model. Figure 2 shows
the available spiking neuron models and their computation times when computed using a fixed,
single-step Euler integration method.

Figure 2 Comparison of Neuron Model Implementation Cost and Biological Plausibility
(Izhikevich , 2004)

Having such a wide range of models leads itself to two approaches when designing a CPG. One
can use as biologically plausible a model as possible and discover what functionality and control
comes with such a model (examples of this approach include Rasche et al (1998) using the
Hodgkin-Huxley model and Simoni and DeWeerth (1999) using the Morris-Lecar model) or one
can use as simple a model as possible and design the extra functionality into the system as needed
(An example of this approach is of Lewis et al (2005) using Integrate and Fire with Spike-
Frequency Adaptation neurons).

The advantages of using the second approach are twofold. Firstly, each neuron model is based on
a differential equation with multiple parameters. As the complexity of the model and the network

14
increases so does the parameter count making it more and more difficult to solve the required
parameters. Secondly, if the CPG is software based and is to be implemented with limited
processing power, having a computationally efficient neuron model means that the equations
governing the central pattern generator can be solved faster allowing more time for other
processing applications or for the processing of a more complicated network.
3.5.2. CPG design using spiking neurons
The simplest CPG that can be built using spiking neurons is the half-centred oscillator (Tenore et
al, 2005), which consists of two neurons driven by the same tonic input and mutually coupled
through inhibition. By themselves these neurons have no rhythmogenic abilities, but when
coupled, they are capable of producing rhythmic outputs. If the inhibition between the two
neurons is equal (i.e. the inhibition signal from neuron one to neuron two is equal to the inhibition
signal from neuron two to neuron one) it is possible to cause a 180 phase difference between
neural outputs, with each neuron firing for 50% of the period. If unbalanced inhibition coupling
strengths are used, it is possible to cause different phase differences and one neuron will fire for a
longer portion of the period than the other neuron.

Figure 3 A half-centred oscillator

Figure 3 shows the layout of a typical half-centred oscillator. Neurons CPG1 and CPG2 have the
same parameters with the same constant tonic input and are reciprocally coupled through
inhibition; that is when one neuron fires, it inhibits the other neuron causing a negative
hyperpolarisation of its membrane potential. Assume that one neuron, say CPG1, will reach its
threshold voltage first. CPG1 will fire, inhibiting CPG2 and causing a negative hyperpolarization
I
1-2
= Inhibitory coupling from CPG1 to CPG2
I
2-1
= Inhibitory coupling from CPG2 to CPG1
I
1-2 I
2-1
CPG1
CPG2

15
of CPG2s membrane potential Both CPG1 and CPG2 will start charging towards their
thresholds, however, CPG1 will reach its threshold first as CPG2 has to recover from its
inhibition. CPG1 will once again inhibit CPG2. This process would continue forever if there was
no spike-frequency adaptation. However, if the neuron model used features spike-frequency
adaptation, eventually the firing rate of CPG1 will decrease such that CPG2 has sufficient time to
charge to its threshold voltage before the next inhibition signal from CPG1. When CPG2 reaches
its threshold voltage an inhibition signal is sent to CPG1 and the cycle repeats itself.

More complex CPGs can be built up by using half-centred oscillators as their building blocks.
This is based on the concept of using one half-centred oscillator per limb or segment to be
controlled and then coupling the different oscillators together to introduce the required phase
shifting between the oscillators.

Lewis et al (2005) used 2 half-centred oscillators coupled together to produce the basic pattern
responsible for coordinating locomotion of a biped robot. This network was then coupled, via
inhibition, to an eight neuron network responsible for driving the patterned movement of the
bipeds hips and knees. Walking patterns were generated by using the following phase
relationships. The left and right hips were 180 out of phase with each other. The hip flexor and
extensor neurons were 180 out of phase with each other and the knee flexor and extensor
neurons were 90 out of phase with the hips flexor and extensor neurons. Lewis et al called this
network the Canonical network and it is shown in Figure 4.

Figure 4 Canonical network design (Lewis et al, 2005)

16
3.5.3. CPG design using non-spiking neuron models
An alternative to CPG design using spiking-neuron models, is to use models that approximate the
mean firing rate of neurons, such as the Matsuoka neuron model. In a network of n Matsuoka
type neurons the i
th
neuron is described as follows

Where, u
i
is the membrane potential of the i
th
neuron, v
i
is used to describe the degree of
adaptation, T
u
, T
v
and are used to specify the time constant for the adaptation, w
ij
describes the
strength of the inhibitory connection between the neurons and u
0
is a constant input signal (Inada
and Ishii, 2004).
3.5.4. Problems with CPG designs using bi-stable oscillators
Marder et al (1997) have shown that CPG networks constructed using bi-stable oscillators, of
which half-centred oscillators are an example of, do not guarantee oscillations and may produce
synchrony.
3.5.5. Alternative approach to CPG design
Eliasmith and Anderson (2000) used an alternative approach in the construction of a Central
Pattern Generator to model Lamprey swimming. This approach avoided using bi-stable oscillators
and thus the problems associated with them. They applied an iterative five step process that
involved creating a system of dynamical differential equations defining the forces to be generated
by the neural system and then redefining the system until it was defined in the space of neural
activities. The resulting network avoided the problems described by Marder et al (1997), The
method proposed by Eliasmith and Anderson (2000), however, has not been tested in designing
controllers for more complicated movements such as bipedal walking.
3.6. Endogenous oscillator neuron approach
Hooper (2000) briefly mentions that an alternative approach to creating rhythmic neural output
through a network of interconnecting neurons is to use endogenous oscillator neurons, however,
no further reading on this concept could be found.
) , 0 max( ,
1
0
i i i i
i v
n
j
j ij i i
i u
u y y v
dt
dv T
u y w v u
dt
du T
= =
+ =

=

17
3.7. Construction of CPGs using non-neural oscillators
Neuron based oscillators are not the only means of constructing central pattern generators. Other
implementations include the use of Phase Oscillators (Cohen et al, 1982; Lewis and Bekey, 2002)
or Van der Pol oscillators. Using these oscillators has the advantage in that they are more open to
mathematical analysis than neuron oscillators. The Van der Pol equation describes self-sustaining
oscillations and has application in the analysis of circuits containing vacuum tubes (Weisstein,
1999 from MathWorld). It is defined as follows
0 ) 1 (
2
= + x x x x & & &
3.8. Conclusions
The following conclusions were drawn from the literature survey:
Simple neuron models are well suited to the design of CPGs to be implemented with limited
processing power.
The Integrate and Fire with Spike Frequency Adaptation is most suited for use in this thesis
as it is computationally efficient, yet its spike-frequency adaptation characteristic allows it to
form a half-centred oscillator and hence a CPG.
It is possible to use a simple network like the Canonical Network shown in Figure 4 to create
the joint angle profiles necessary to realise bipedal walking.

18
4. BACKGROUND TO GENETIC ALGORITHMS
4.1. What is a Genetic Algorithm?
A Genetic Algorithm (GA) is an adaptive stochastic optimisation algorithm involving search and
optimisation that is based on the ideas of evolutionary biology and genetics. These ideas include:
natural selection, inheritance, mutation and crossover. Genetic algorithms were first used by
Holland in 1975 and provide better performance than typical optimisation methods (e.g. linear
programming) in searching large parameter spaces or with fitness functions that are not smooth.

Genetic algorithms use randomization to search very large parameter spaces; however, the
searching done by the algorithm is by no means random in its method. GAs use past
performances to direct the search towards better performing areas of the parameter space, and in
doing so are able to search enormous solution spaces effectively

The algorithm starts by creating a random population of solutions for the problem, however, if an
area in the parameter space is thought to give optimal solutions then the initial population may be
seeded with solutions in that area. Seeding of the initial population may help the algorithm
converge on a final solution in fewer generations than if the initial population was completely
random. Each possible solution in the population is called a chromosome and each variable in the
chromosome is called a gene.

The chromosomes are then trialled on the problem to be solved and their performances are
evaluated using a fitness function. A second generation of solutions is then evolved from the
initial populations best solutions. The evolution of the next generation involves the breeding of
the best parent solutions by cross-over (gene swapping), and random mutation. Some algorithms
use an elitist factor to ensure that the best solutions from the parent generation propagate
undisturbed into the next generation. As the population size for the GA is constant, each new
generation is constructed by weeding out the poorest solutions, of the previous generation, to
make room for the new solutions made from the stronger parents. This is analogous to survival of
the fittest in nature.

19
4.2. Increasing the rate of convergence
Speed of convergence of the algorithm is a critical factor when using GAs. De Garis (1990)
showed that the evolution of a controller can be affected by the starting parameters of the search
and that it is possible to reduce the speed of the search by using two fitness functions. The first
fitness function is more easily satisfied and is used to pre-evolve the solution to an appropriate
level. Once the solution reaches this level, the second fitness function is used to further improve
the solution. De Garis named the above concept Behavioural Memory.
4.3. Types of Genetic Algorithms
There are numerous variations and types of genetic algorithms; however, discussing them all is
outside the scope of this thesis. The two algorithms that were used in this thesis are briefly
discussed.
4.3.1. NSGA-II
This is a multi-goal optimisation method that creates a Pareto front for the objectives. NSGA-II
was developed at the Kanpur Genetic Algorithm Laboratory and aims to minimise the fitness
function.
4.3.2. FBGAMIN
FBGAMIN is an Adaptive Mutation Breeder Algorithm (AMBA), used to optimise single goal
functions. Breeder algorithms are well suited for the optimization of real-valued function inputs,
and the adaptive mutation characteristic of the algorithm utilises a simple feedback loop within
the algorithm to ensure that the mutation rate is always at its optimum. FBGAMIN was developed
at the University of Cape Town and aims to minimise the fitness function. It uses elitism to
ensure that the best solution propagates, unchanged, to the next generation.
4.4. Applications of Genetic Algorithms to CPG networks
Lewis et al (1992) used genetic algorithms to optimise the weights in a network of neurons to
control a 12 degree of freedom hexapod robot. In order to improve the convergence rate of the
GA staged evolution was used. The network consisted of 1 oscillator per leg, with two neurons
per oscillator. One neuron controlled the swing of the leg and the other the elevation of the leg.
The first stage of evolution was to evolve the parameters necessary to create oscillations in a
single two neuron network and hence a stepping motion of the leg. Once the oscillator reached

20
the desired criterion, the optimised parameters were applied to all oscillators in the network. The
second stage of evolution involved finding the connections between the oscillators, so as to
provide coordination of the legs and the generation of a successful walking pattern. The fitness
function used for the second stage of evolution rewarded both forward and backward movement
in straight line. Lewis managed to evolve both the tripod and wave gaits for the robot; however, if
further generations were computed the tripod gait always dominated the wave gait as it was the
most efficient gait for the robot.

Wiley used genetic algorithms to first evolve standing and then walking patterns for a simulated
quadruped. A reduced CPG model using square wave oscillators was used as the input to four
distinct neural networks, where each network was responsible for controlling one leg. The
frequency of the oscillator signals and the weights of the neural network were then solved using a
genetic algorithm. The fitness function was proportional to the distance moved by the quadruped
in a set direction during a specified time period. The terrain and ground friction were altered for
each experiment, and controllers capable of walking were evolved specific to each situation.

Inada and Ishii (2004) used the Matsuoka neuron model to build a CPG to control a simulated
biped with ankles, knees, hips and pitching of the waste. They used genetic algorithms to
optimise the network parameters and the targets for the algorithms were obtained by real-time
video analysis of the angles of the ankle, knee, hip and waste joints. A staged evolution approach
was used to break down the search for parameters into more manageable sub sections.
4.5. Conclusions
The following conclusions were drawn from the literature survey:
Genetic algorithms are a useful tool in solving the parameters of the CPG network.
Genetic algorithms can be used to evolve controllers for walking.
Speed of convergence is a critical issue when using genetic algorithms.
Genetic algorithms are more effective if the principles of Behavioural Memory and
Staged Evolution are applied.
21
5. THE DEVELOPMENT AND SIMULATION
ENVIRONMENT

A simulation environment was created to aid the development of the walking algorithms and to
simulate the resultant walking patterns. The simulation took the form of a Graphical User
Interface (GUI) that took in a vector of joint angle trajectories. The GUI would then display a
graphical representation of legs walking using the given trajectories, as well as calculate the
accelerations, velocities and torques needed to move each of the joints.
5.1. Conventions and assumptions
Figure 5 shows the naming convention for the RedBots limbs and angles. The hip angles are
defined by taking the angle from a vertical line through the hips to the thigh of the relevant leg. If
the angle is taken in the direction of movement (the right leg in Figure 5) then the angle is
positive and if the angle is taken against the direction of movement (the left leg in Figure 5) then
the angle is negative.

Figure 5 Naming conventions for RedBot simulation

Right hip angle
Left hip angle
(negative)
Right knee
angle
Right foot
angle
Left knee
angle
Left foot
angle
Ground
Right thigh
Right shin
Left thigh
Left shin
Y
X
Direction of movement
z direction is out of the page
Z
22
The knee angles were calculated by taking the angle from the thigh to the shin, on the back of the
leg. Thus, if the knee angle is 0 then the shin will be bent up against the thigh and if the angle is
180 then the leg will be straight.

The above angle conventions are used through out this thesis project.

The RedBot had not arrived at the time the simulation was being coded, and thus a number of
educated guesses and assumptions were taken where necessary to build the simulation. The
following assumptions and conventions were made:
The RedBot is supported in the sagittal plane (x-y plane of Figure 5). Thus it was decided
to assume there is no movement of the RedBot in frontal plane (y-z plane of Figure 5)
and the system was reduced to a two dimensional system.
The support for the RedBot is attached to the RedBots waist. It was assumed that
orientation of the hips remains fixed and that they are unable to rotate about the support.
The simulation assumed 100% friction between the stance foot and the floor i.e. there is
no slippage between the stance foot and the walking surface.
Motion of the hips was only allowed in the y-direction. Movement of the hips in the x-
direction was constrained. Thus when walking was simulated the RedBot appeared to be
walking in place on a treadmill.
For the calculations of inertias, moments and torques, all joints were assumed to be rigid,
except for the joint under consideration. Together with the assumption that there is 100%
friction between the stance foot and the ground, the section between the stance foot and
the joint under consideration could be considered to be fixed to the ground.
The lower foot was assumed to be the stance foot
The feedback values for the RedBot were assumed to be the last positions the joints were
told to go to.
The mass of the plastic from which the RedBot was constructed was assumed to be
negligible.
Each limb was approximated as a mass less beam with a single point mass somewhere
along its length.
The mass of each thigh and shin was modelled as being the mass of the servo motor
mounted in it.
The mass of the hips was modelled as being the mass of the supporting pole.
23
The distances of the point masses away from the different joints were assumed to be half
the height of the servo motor used to actuate that joint.

The servo characteristics were obtained from their data sheets and are shown in detail in Table 1
of Chapter 6. The thighs and shins of the RedBot are 100mm long and the supporting hip section
80mm tall.

The above assumptions and parameters resulted in the following model of the RedBot.

Figure 6 RedBot weights and lengths used in the simulation

5.2. The simulation process and calculations
In order to simulate the walking motion of the RedBot the simulation accepted a series of vectors
that described the joint angles of the RedBot at each point in time. The inputted angles were used
to calculate the x and y coordinates of each hip, knee, foot and point mass relative to the hips. The
x and y coordinates of the knees and feet were used to create a visualisation of the RedBot
walking, whilst the x and y coordinates of the point masses were used to calculate the necessary
moments of inertias and torques.

As mentioned above when calculating inertias, moments and torques all joints were assumed to
be rigid except for the joint under consideration, resulting in the section of the RedBot between
Ground
l
1
l
2
l
1
l
2
l
1
l
1
l
3
l
3
m
1
m
2
m
2
m
3
m
3
l
1
=100mm
l
2
=17.5mm
l
3
=20mm
m
1
=100g
m
2
=55g
m
3
=31g
24
the joint under consideration and the ground being assumed to be fixed to the ground. For
example, consider the scenario in which the right leg is the stance leg and the right knee is being
examined. The right knee servo motor is then responsible for rotating the hips and the entire left
leg about the right knee. If, however, the left knee was under consideration then the left knee
servo motor would only be responsible for rotating the left shin about the left knee.

Figure 7 Torque calculation flow diagram

The following is a general method used to find the torques acting on a single joint.

The torque due to gravity on the joint was calculated as follows.
=
masses all
n x n gravity
Mg l
) ( ) (

Where is the torque, l
x
is the displacement between the mass and joint in the x-direction and g is
the acceleration due to gravity.

The angular accelerations needed to move the limbs in the desired manner were calculated as
follows:
2
1 1
2
t
n n n
n
+
=
+

where is the angular acceleration, is the angular position and t is the time step between
samples.
Next angles from walking algorithm Current joint angles of RedBot
Calculate net torques to move limbs
Calculate accelerations to move limbs Calculate inertia of limbs
Calculate torques from gravity on
limbs
Calculate torques required from servos
25

The rotational inertia around each joint was calculated using the following equation
=
masses all
n n
Mr I
2

Where I is the rotational inertia, M is the mass of each limb and r is the distance between the
mass and the joint.

The net torque acting on a joint was calculated as follows:
n n n net
I =
) (

Once all the above calculations had been performed it was possible to calculate the torque
required by the servo motor to actuate the specific joint as follows:
) ( ) ( ) (
) ( ) ( ) (
n gravity n net n motor
n gravity n motor n net

=
+ =

The development GUI applied the above calculations to each joint at each sample in the vector of
joint angle trajectories. The GUI allowed the different torques and accelerations needed to
implement the joint angle trajectories to be plotted against time. This allowed for a comparison
between the torques needed and the torques available from the servo motors.

It was also desired to plot the angular velocities of each joint. The angular velocity of a single
joint was calculated as follows:
t
n n n n
n
+
=
+ +
12
8 8
2 1 1 1

Where is the angular velocity, t is the time step between samples and is the angular position

5.3. The final development GUI
Besides being able to calculate the torques, accelerations and velocities of all the joints, the GUI
was extended to aid in the development of the different algorithms. Parameters relating to the
different algorithms could be altered through the GUI, and plots, such as neuron membrane
potentials for the CPG algorithm, could be generated. The GUI allowed different sets of
26
parameters to be saved and loaded and the simulation of the walking of the RedBot could be
converted to a movie file for presentation purposes.

Figure 8 The development and simulation GUI

The development GUI was never intended for use by anyone other than those who created it. It
was merely a tool used to ease the development of the Central Pattern Generator and If-Then rule
algorithms.

27
6. HARDWARE
6.1. Introduction to the RedBot
The robot platform used in this thesis is the RedBot biped manufactured by AlegRobotics Inc.
in the United States of America. The RedBot stands 28cm tall and has 2 degrees of freedom (hip
and knee joints) per leg. For feet the RedBot has stumps and it is capable of walking between
speeds of 0.5Hz and 3Hz. The RedBot cannot run.

Figure 9 The RedBot

Motion of the RedBots limbs is controlled by servo motors. There is one servo motor per joint.
Table 1 shows the type of servo motor used in each joint as well as the servo motors weight, size
and performance characteristics.

Table 1 Joint Servo Motor Names and Characteristics
Maximum Torque (N.m) Speed (s/60) Joint Servo Motor
4.8V
Supply
6.0V
Supply
4.8V
supply
6.0V
supply
Weight
(grams)
Size
(wlh(mm))
Right Hip Futaba S9402 0.628 0.785 0.13 0.10 55 204138
Left Hip Futaba S9402 0.628 0.785 0.13 0.10 55 204138
Right knee Futaba S9602 0.256 0.265 0.11 0.09 31 351530
Left Knee Futaba S9602 0.256 0.265 0.11 0.09 31 351530

28
With a supply voltage of 6V the servo motors give better performance in terms of both speed and
torque than at 4.8V. It was thus decided to run the servos with a 6V supply.

Motion of the RedBot is constrained in the sagittal plane through an aluminium pole attached to
the RedBots hip on one end and to a pivot on the other end. This prevents the RedBot from
falling over in the frontal plane whilst still allowing it to fall over in the sagittal plane.

Joint angle feedback was achieved by tapping into the potentiometers used by the servo motors
internal circuitry for position control. Binary foot and shin switches were added to the RedBot to
detect its state for use by the rules based controller.
6.2. The microcontroller
Each servo motor on the RedBot is controlled using a Pulse Width Modulated (PWM) signal. To
reduce the hardware needed to control the RedBot it was decided to use the microcontroller, and
not external circuitry, to generate the PWM signals. It was also decided that the microcontroller
would have to have provision for the addition of ankle joints to the RedBot at a later stage.

The requirements for the microcontroller are shown below
Table 2 Requirements for microcontroller used to control the RedBot
Item Quantity Reason
PWM
channels
6 One channel to control each hip and knee servo motor with an
extra two channels for the later addition of ankles
ADC
channels
8 Four channels to obtain position feedback from the hips and
knees, two channels for the future inclusion of ankles position
sensors and two channels for the possibility of foot pressure
sensors
IO Pins 4 To allow for the addition of simple binary foot switches and other
logic based IO
UART 1 To allow communications with the host PC.

The dsPIC30F4011 fulfilled all the above criteria and was chosen for use in this thesis. In
addition to this, because the dsPIC30F4011 is designed for DSP applications it is able to execute
all instructions (excluding instructions that change the program flow) in a single cycle.

29

The following table shows which pins of the dsPIC30F4011 were used for what application
Table 3 dsPIC30F4011 pin layouts
PWM signal pin layout
Servo Motor Pin Name Pin Number
Left hip OC1 23
Right hip OC2 18
Right knee OC3 22
Left knee OC4 19
Servo motor angle sensor pin layout
Servo Motor Pin Name Pin Number
Left hip AN2 4
Right hip AN3 5
Right knee AN4 6
Left knee AN5 7
Binary switch pin layout
Switch Pin Name Pin Number
Right foot - 28
Left foot - 27
Right shin - 9
Left shin - 10

6.3. RedBot to dsPICDEM2 connection board
A basic connection board was built to interface the RedBot to the dsPICDEM2 development
board. The connection board was powered from a 6V DC power supply (6A max current) and
contained the necessary power connections for the RedBots servo motors. The board also
connected the PIC to the PWM or feedback connections on the RedBot. Since the supply was 6V,
it was necessary to scale all feedback signals from the 0-6V range to the 0-5V range to be
compatible with the PICs input pins. This was done by using a simple resistor divider network.
The inputs to the ADC pins were buffered using op-amps as unity gain voltage followers. The
LM324 quad op-amp was used, and thus only one chip was required to buffer all the angle sensor
feedback signals.
30
7. PRELIMINARY METHODS

Before the construction of the Central Pattern Generator could commence it was necessary to
perform a number of smaller tasks. These tasks included: programming a method to solve
differential equations, understanding the principles of tonic spiking and spike frequency
adaptation and obtaining human walking data to use as a reference for the outputs of the CPG.
7.1. Extracting human walking data from Gaitlab
Since the aim of this thesis was to create a controller for biped walking it was useful to have
human hip and knee angle trajectory data available for inspection and manipulation on the
computer. The data was extracted from the file peak.dst in Gaitlab.

The following steps were taken obtain the data:
1) Gaitlab was run and the file Peak.dst was opened.
2) Peak.dst was then processed running all procedures.
3) The resulting data was then browsed and the following sets of data were copied and
imported into Microsoft Excel: JOINTANGLE LEFT HIP FLEXEXT, JOINTANGLE
RIGHT HIP FLEXEXT, JOINTANGLE LEFT KNEE FLEXEXT and JOINTANGLE
RIGHT KNEE FLEXEXT.
4) The data from the hips and knees was then scaled to correspond to the convention
described in 5.1.
5) The hip angles and scaled knee angles were then plotted.
7.2. 4
th
Order Runge-Kutta method
The neuron models described by Izhikevich are all based on differential equations, it is thus
necessary to have a numeric method capable of solving the differential equations. The 4
th
Order
Runge-Kutta was chosen for this function because it is simple to implement and has relatively
high accuracy.

The aim of this experiment was to use the Runge-Kutta algorithm (given below) to solve the
differential equation, x
dt
dx
= where x(0) = 1 using a time step of h = 0.002s,. The results of the
31
Runge-Kutta were then graphically compared to the actual solution of the equation which
is
t
e x = .

The 4
th
Order Runge-Kutta algorithm is as follows:

Given an equation ) , (
) (
x t f
dt
t dx
= then if at any time, t
i
,the current value of x ,x
i
, is known, it is
possible to solve the next value of x, x
i+1
, at t
i+1
=t
i
+h as follows:
( ) [ ]
) , (
)
2
1
,
2
1
(
)
2
1
,
2
1
(
) , (
2
6
3 4
2 3
1 2
1
4 3 2 1 1
hm y h t f m
hm y h t f m
hm y h t f m
x t f m
m m m m
h
x x
i i
i i
i i
i i
i i
+ + =
+ + =
+ + =
=
+ + + + =
+

7.3. Simulation of a single neuron
Before experiments with the Central Pattern Generator as a whole could begin, it was necessary
to have an idea of how the Integrate and Fire with Spike Frequency Adaptation (IFA) neuron
behaves. The aim of this experiment was to build a flexible framework for the Central Pattern
Generator, using IFA neurons, that was easy to adapt for the inclusion of additional neurons or
coupling connections. The framework was then used to investigate the effects of the different
neuron parameters on the neurons behaviour and to find parameters that illustrate the principles
of tonic spiking and spike frequency adaptation. The parameters for the neuron were found by
trial and error.

The IFA neuron model used is based on that given by Izhikevich (2004). If there are n neurons in
a network then the i
th
neuron is defined as follows:

32
i
i
i thresh i
i i i
n
i j j
i j i
g t e
g
c v then v v if
v d g bv a I v
=
= >
+ + =

=

) (
) (
, 1

Where, v
i
is the membrane potential (a voltage) of the i
th
neuron in the network, a (a current) is
the tonic input of the neuron,
=

n
i j j
i j
I
, 1
is the sum of the inhibition signals (currents) from other
neurons in the network, b is the neurons membrane impedance, v
thresh
is the membrane potential
threshold voltage (when v exceeds v
thresh
the neuron is said to fire), c is the reset voltage of the
neuron, g is the potassium activation gate of the model or adaptation characteristic, which
increases by e/ every time the neuron fires via the Dirac Delta function , is the adaptation time
constant and d changes the weight of self-inhibition.

Since it is not possible to physically realise the Dirac Delta function the above model was
implemented as follows:
i
i
inc
inc i i i
thresh i
i i i
n
i j j
i j i
g
g
el above the in
t e
replaces g where
g g g and c v then
v v if
v d g bv a I v
=
+ = =
>
+ + =

=

mod
) (
) (
, 1

A detailed algorithm showing the framework of the CPG as well as how the above neuron model
is applied within this framework is given below.

33
Set n= to number of neurons in network
Create nxm array called params to store neuron variables.
m = n +8, which allows the array to store all parameters necessary for the IFA model as well as the time and the ability
for the i
th
neuron to receive coupling signals from the other n-i neurons in the network
Initialize all variables. All membrane potentials and potassium concentrations to be set to 0
Loop for desired simulation time period in steps of 2ms
Loop for i = 1 to n
(i.e. through all neurons in network)
Update membrane potential, current time and potassium concentration of i
th

neuron in network by passing i
th
row of params to IFA
Is v
i
>vthresh?
v
i
=c
Set inhibition signals to other neurons to
desired values
Set inhibition signals to other
neurons to 0
Looped through all neurons?
Add current membrane potential and potassium
concentration to i
th
row of xv and gv respectively
Time period up?
Manipulate and/or plot xv, gv and spikev as
desired
Yes
No
Yes
Yes
No
No
Create vectors xv, gv and spikev with n rows each to store the neuron membrane potentials, potassium concentrations
and whether each neuron has fired or not (1 = yes, 0 = no) respectively for each time step in the simulation.
Add 1 to i
th
row of spikev
Add 0 to i
th
row of spikev
CPGFRAMEWORK 1
34

Figure 11 Algorithm for the IFA function

The following steps were taken to find neuron parameters that demonstrate tonic spiking and
spike frequency adaptation:
The n term in CPGFRAMEWORK1 was set to 1 as only one neuron was to be simulated.
The time period for the simulation was set to 3s.
The a term of the IFA model was held at zero until t = 0.5s, at which point it was set to
its desired value. At t=2.5s it was set back to zero.
Iterations of the simulation were run plotting xv, gv and a against time until satisfactory
values of a, b, c d, , g
inc
and v
thresh
were found.
Initialise all variables using constants and
passed parameters (time step h = 0.002s)
=

=
n
i j j
i j
I I
, 1

Increment the time by h
Get next value of the membrane potential using dx=a-bx+I-g(x-d)
and the Runge-Kutta algorithm described previously
v > v
thresh
?
Get next value of the potassium concentration using
g
dg

=

and the Runge-Kutta algorithm
g = g + g
inc

Return updated v, g and t values to calling function
No
Yes
35
8. PRELIMANRY RESULTS
8.1. Extraction of human walking data
The following graph shows the joint angle profiles extracted from Gaitlab.
-50
0
50
100
150
200
0 0.5 1 1.5 2
Ti me (s)
A
n
g
l
e

(
D
e
g
r
e
e
s
)
Left Hip
Right Hip
Left Knee
Right Knee

Figure 12 Hip and Knee Angle Trajectories during normal human walking

By inspecting the graph it is possible to see that there are slight differences between the right and
left legs joint angle profiles. There are many possible reasons for this including:
The human body is not perfectly symmetrical and as a result the gait will not be perfectly
symmetrical.
The surface on which the test subject walked may not have been perfectly level.
The test subject may have had a slight injury causing an unsymmetrical gait.

The joint angle trajectories for the RedBot, however, are to be identical but 180 out of phase. It
was thus decided to use only one legs joint trajectories as a reference. The left leg was arbitrarily
chosen as the reference.

Note that the knee angle profiles contain a double bump. The flexion stage between the two
extension peaks cushions the effect of the foot on landing (Trew and Everett, 1998).

36
8.2. 4
th
Order Runge-Kutta results
The 4
th
order Runge-Kutta algorithm described above was successfully implemented in
MATLAB 7. Figure 13 below compares the solution of x
dt
dx
= as found by the Runge-Kutta
method against the actual solution of the equation.
0 0.2 0.4 0.6 0.8 1
1
1.5
2
2.5
3
Time (s)
F
u
n
c
t
i
o
n

v
a
l
u
e
Actual solution (x =exp(t))
Runge-Kutta solution

Figure 13 Runge-Kutta results vs. actual solution

Figure 13 shows that the Runge-Kutta solution closely approximates the ideal solution for small
values of t. This is acceptable because the IFA neuron equations are cyclic in nature and reset
themselves every time the neuron fires, and thus the equation will only be solved over small
values of t. A smaller h value would result in a more accurate approximation, however, it would
also result in an increase in the number of iterations necessary to solve for a given time period,
which increases the processing time of the algorithm. A larger h value would decrease the
accuracy of the approximation but also decrease the processing time needed to simulate a given
time period.

37
8.3. Simulation of a single neuron results
CPGFRAMEWORK1 and IFA were successfully implemented in MATLAB. By experimentation
with each of the parameters within the model a broad idea of what each parameter did was
gained, however, exact relationships between the parameters and the neurons firing rate could
not be established. The basic affect of each parameter on the neurons behaviour is described
below:
c is the value to which the neurons membrane potential resets itself to after the neuron
has fired.
v
thresh
is the threshold voltage of the neurons membrane potential. When the membrane
potential reaches v
thresh
the neuron fires and resets its membrane potential to c.
An increase in a causes an increase in the neurons charging rate, resulting in the neuron
reaching its threshold voltage quicker and increasing its firing rate.
An increase in b causes an increase in the rate of exponential decay of the neurons
membrane potential, slowing the rate of charge of the neurons membrane potential and
decreasing the rate of firing of the neuron.
An increase in g
inc
causes a larger increase in the neurons potassium concentration every
time the neuron fires, and as a result the neurons firing rate slows down more quickly,
An increase in increases the time taken for the potassium concentration of the neuron to
discharge.
Making d more negative increases the self-inhibition of the neuron slowing its firing rate.

To mathematically analyse the model to obtain exact relationships for a single neuron may have
been possible, however, to form the CPG, each neuron would have to be coupled to a larger
network of neurons. It would thus be necessary to analyse the entire network to accurately predict
the results of a single neuron. This is a huge undertaking and was thus not performed.

38
Numerous iterations of trial and error were required to find parameters that clearly demonstrate
the principles of tonic spiking (Figure 14) and spike frequency adaptation (Figure 15). The exact
parameter values used in these two examples can be found in Appendix 1.

0 0.5 1 1.5 2 2.5 3
0
0.5
1
Time (s)
M
e
m
b
r
a
n
e

P
o
t
e
n
t
i
a
l
Toni c Spi ki ng of Neuron
0 0.5 1 1.5 2 2.5 3
0
0.05
0.1
0.15
0.2
Time (s)
K

C
o
n
c
e
n
t
r
a
t
i
o
n
Spi ke Frequency Adaptati on Characteri sti c of Neuron
0 0.5 1 1.5 2 2.5 3
0
10
20
30
Time (s)
T
o
n
i
c

I
n
p
u
t
Neuron Toni c Input

Figure 14 Results of Neuron Simulation experiment to illustrate tonic spiking
0 0.5 1 1.5 2 2.5 3
0
0.5
1
Time (s)
M
e
m
b
r
a
n
e

P
o
t
e
n
t
i
a
l
Toni c Spi ki ng of Neuron wi th Spi ke Frequency Adaptati on
0 0.5 1 1.5 2 2.5 3
0
0.05
0.1
0.15
0.2
Time (s)
K

C
o
n
c
e
n
t
r
a
t
i
o
n
Spi ke Frequency Adaptati on Characteri sti c of Neuron
0 0.5 1 1.5 2 2.5 3
0
10
20
30
Time (s)
T
o
n
i
c

I
n
p
u
t
Neuron Toni c Input

Figure 15 Results of Neuron Simulation Experiment Showing the Neurons Spike Frequency
Adaptation Characteristic
39
9. TRIAL AND ERROR CPG METHODS

All neurons used in this chapter are of the Integrate and Fire with Spike Frequency Adaptation
type, as defined by Izhikevich (2004), unless stated otherwise. The 4
th
order Runge-Kutta
algorithm, described above, with a time step of 0.002s is used to solve all differential equations in
the networks.
9.1. Construction of a Half-Centred Oscillator
Half-centred oscillators are responsible for keeping time of the central pattern generator. The aim
of this experiment was to find parameters, through trial and error, that cause the half-centred
oscillator, shown below, to oscillate at 0.5Hz and then through further trial and error of the
inhibitory coupling strength find the coupling weights to cause oscillations at 1Hz, 1.5Hz, 2Hz,
2.5Hz and 3Hz

Figure 16 the half-centred oscillator layout

CPGFRAMEWORK1 described in 7.3 was used for this experiment and the following steps were
taken:
1) n was set to 2
2) The time period for the simulation was set to 10s
3) c was set to 0 for all iterations of the simulation.
4) Iterations of the simulation were run plotting xv and spikev against time, whilst altering
the values of a, b, d, , g
inc
, v
thresh
, I
1-2
and I
2-1
to find the necessary parameters to cause
oscillations at the desired frequencies.
Inhibitory
Coupling
I
1-2 I
2-1
CPG1
CPG2
40
9.2. Refining the neuron model
In order to make the search for parameters an easier process and to enforce steady state
oscillations on the half-centred oscillator as quickly as possible, it was decided to reset the
potassium concentration of the neurons to zero if they did not fire for a specified period of time.
This new neuron model was called IFA2 to distinguish it from the standard IFA model. The new
equations are defined below:
0
) (
) (
, 1
= >
=
= >
+ + =

=

i reset fire
i
i
i thresh i
i i i
n
i j j
i j i
g then t t if
g t e
g
c v then v v if
v d g bv a I v

Or for implementation
0
) (
, 1
= >
=
+ = =
>
+ + =

=

i reset fire
i
i
inc i i i
thresh i
i i i
n
i j j
i j i
g then t t if
g
g
g g g and c v then
v v if
v d g bv a I v

Where all the parameters are the same as before, but t
fire
and t
reset
are the time elapsed since the
neuron last fired and the maximum time allowed to elapse before the potassium concentration is
set to zero, respectively.

The algorithm showing how the IFA2 model is implemented is shown in Figure 17. The
algorithm for IFA remains the same as that shown previously in Figure 11.

The algorithm was then tested by applying the parameters obtained from 9.1 to the new algorithm
and neuron model. If the new model was implemented correctly then the half-centred oscillator
should oscillate at just below 0.5Hz and the frequency of oscillations should not change over
time.

41
The parameters used for this experiment are shown in the table below.
Table 4 Parameters to test the IFA2 neuron model
a b d g
inc
v
thresh
c I
1-2
I
2-1
t
reset
(s)
10 20 -25.94 0.015 1.63 0.4 0 -2000 -2000
0.3

42
Set n= to number of neurons in network
Create nxm array called params to store neuron variables.
m = n +8, which allows the array to store all parameters necessary for the IFA2 model as well as the time and the ability for the
i
th
neuron to receive coupling signals from the other n-i neurons in the network
Initialize all variables. All membrane potentials and potassium concentrations to be set to 0
Loop for desired simulation time period in steps of 2ms
Loop for i = 1 to n (i.e. through all neurons in network)
th

th
Is v
i
>vthresh?
v
i
=c
desired values
Set inhibition signals to
other neurons to 0
Add current membrane potential and potassium
concentration to i
th
row of xv and gv respectively
Time period up?
Manipulate and/or plot xv, gv and spikev as
desired
Yes
No
Yes
Yes
No
No
Create vectors xv, gv and spikev with n rows each to store the neuron membrane potentials, potassium concentrations and
whether each neuron has fired or not (1 = yes, 0 = no) respectively for each time step in the simulation.
Add 1 to i
th
row of spikev
Add 0 to i
th
row of spikev
CPGFRAMEWORK2
Create nx1 array called tfire to store the time since each neuron last fired
Increment i
th
row of tfire by 0.002s
tfire(i) = 0
tfire(i) > t
reset
?
g
i
=0
No
Yes
Initialise CPG
43
9.3. Using Genetic Algorithms to aid trial and error
In order to get a good filtered shape of the neuron spike trains, it is desirable for the neurons to
start firing at a very rapid frequency and then slow down to a very slow frequency. It would be
very useful to have a tool to aid in the search of parameters that could cause the neuron to fire in
the specified manner.

The aim of this experiment was to break the time period for which a neuron fires, during
oscillations at a specified frequency, into smaller time periods and then to see if it is possible to
specify the number of times a neuron fires in each specific time period, using the NSGA-II
Genetic Algorithm to solve the necessary parameters.

It was decided to use a frequency of 1.5Hz as the basic frequency for this experiment. The time
period for which the neuron fired over was thus 0.333s (half a period). 1.5Hz was chosen because
it is approximately midway between the RedBots slowest walking frequency (0.5Hz) and fastest
walking frequency (3Hz).

NSGA-II is a multi-goal optimisation algorithm and it thus needs one fitness function per goal.
The functions that take in the parameters to be optimised and return the fitness obtained from the
fitness function are called objective functions.

The goals for this experiment were as follows:
The neuron must fire 15 times in the first half of its spiking pattern
The neuron must fire 5 times in the second half of its spiking pattern

CPGFRAMEWORK2 (with n set to 1) was used as the basic algorithm for each of the objective
functions. The values of v
thresh
, c, b and t
reset
were predefined to 0.4, 0, 50 and 0.2s respectively
and only a, g
ginc,,
d and were optimised using NSGA-II.

The first objective function had a simulation time period of 0.168s. Once the simulation had
completed running the number of times the neuron fired during the simulation time period was
calculated from spikev and stored in numspikes. The fitness of the solution was calculated using
the following fitness function:
fitness = (15-numspikes)
2

44

The second objective function had a simulation time period of 0.334s. The number of times the
neuron fired between 0.168s and 0.334s (0.333 is not a multiple of 0.002) was calculated from
spikev and stored in numspikes. The fitness of the solution was calculated using the following
fitness function:
fitness = (5-numspikes)
2

The objective functions names were then added to the evaluate_objective function of the NSGA-
II algorithm and the NSGA-II function was run as follows:
The number of generations was set to 10 and the population size was set to 20.
The number of objective functions was set to 2 when prompted
The number of evaluation variables was set to 4 when prompted
The limits for the variables to be solved are defined below
Table 5 Maximum and minimum parameter values for
nsga_2 Function
Parameter Min Value Max Value
a 0 200
g
inc
0 2
d -100 0
0 5

The above experiment was repeated 4 times.
45

10. TRIAL AND ERROR CPG RESULTS
10.1. Half-Centred Oscillator results
Parameters to cause steady state oscillations at the desired frequencies could not be found through
trial and error. Parameters (shown in Table 6) that were capable of causing an initial oscillating
frequency of just below 0.5Hz could be found, however, the frequency of oscillation increased
every cycle until a steady state frequency of 0.63Hz was reached at 7.6s.

The increase in frequency is a result of the potassium concentrations of the two neurons not being
able to discharge fully during the neurons periods of inactivity as shown in Figure 20.
Parameters for the neuron could not be found that caused the potassium concentrations to
decrease to zero within 1s and still cause a frequency of oscillation of 0.5Hz. To run the
simulation for multiple periods until steady state is reached is also not desirable because it is a
very time consuming process. It was thus decided to abandon this experiment and instead make
changes to the IFA neuron model to make it easier to find parameters that give the desired
behaviour. These changes are discussed in section 9.2.

0 2 4 6 8 10
-4.5
-4
-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
Time (s)
N
e
u
r
o
n

M
e
m
b
r
a
n
e

P
o
t
e
n
t
i
a
l
CPG1
CPG2
0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
1
Time (s)
S
p
i
k
e

V
a
l
u
e

(
V
o
l
t
s
)
CPG1
CPG2
Figure 18 Membrane Potentials of a half-centred
oscillator with an initial frequency of 0.5Hz
Figure 19 Firing patterns of a half-centred
oscillator with a starting frequency of 0.5Hz
46
0 2 4 6 8 10
0
0.02
0.04
0.06
0.08
0.1
Time (s)
P
o
t
a
s
s
i
u
m

(
K
)

C
o
n
c
e
n
t
r
a
t
i
o
n
CPG1
CPG2

Figure 20 Potassium concentrations of a half-centred oscillator with a starting frequency of 0.5Hz

Table 6 Parameters to cause oscillations of a half-centred oscillator at a steady state frequency of
0.63Hz
Frequency
of
oscillation
(Hz)
a b d g
inc
v
thresh
c I
12
I
21
0.63 10 20 -25.94 0.015 1.63 0.4 0 -2000 -2000

10.2. Refining the neuron model results
The IFA2 neuron model worked as predicted. Using the parameters obtained in 9.1 and shown in
Table 6, oscillations began at just below 0.5Hz and did not change over time. The results can be
clearly seen in the figures below. Attention should be paid to Figure 23 which shows the
potassium concentrations of CPG1 and CPG2 resetting to zero if the specific neurons do not fire
for more than 0.2s.
47
0 2 4 6 8 10
-4.5
-4
-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
Time (s)
N
e
u
r
o
n

M
e
m
b
r
a
n
e

P
o
t
e
n
t
i
a
l
CPG1
CPG2
0 2 4 6 8 10
0
0.2
0.4
0.6
0.8
1
Time (s)
S
p
i
k
e

V
a
l
u
e

(
V
o
l
t
s
)
CPG1
CPG2
Figure 21 Membrane potential of a half-centred
oscillator, using IF&A2 neurons, at 0.5Hz
Figure 22 Firing patterns of a half-centred
oscillator, using IF&A2 neurons, at 0.5Hz
0 2 4 6 8 10
0
0.02
0.04
0.06
0.08
0.1
Time (s)
P
o
t
a
s
s
i
u
m

(
K
)

C
o
n
c
e
n
t
r
a
t
i
o
n
CPG1
CPG2

Figure 23 Potassium concentrations of a half-centred oscillator, using IFA2 neurons, at 0.5Hz

By not having to take the time taken for the potassium concentrations to discharge into account,
and because steady state oscillations are reached almost immediately, the search for parameters to
get the neurons to behave as desired was greatly simplified.
48
10.3. Using Genetic Algorithms to aid trial and error results
The target for this experiment was for the neuron to fire 15 times in the first 0.168s and 5 times in
the next 0.166s. The results of the four iterations of this experiment are shown in Figure 24 to
Figure 27 . NSGA-II could not find parameters to realise the desired firing pattern exactly. The
closest result to the desired one (Solution 17 of the third trial) was for the neuron to spike 13
times in the first 0.168s and 4 times in the next 0.166s.

0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
N
u
m
b
e
r

o
f

S
p
i
k
e
s
1 3 5 7 9 11 13 15 17 19
Sol uti on
t<0.168s 0.168s <t<0.334s

0.00
5.00
10.00
15.00
20.00
25.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Solution
N
u
m
b
e
r

o
f

S
p
i
k
e
s
t<0168s 0.168s <t <0.334s

Figure 24 Results of NSGA-II first trial Figure 25 Results of NSGA-II second trial
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Solution
N
u
m
b
e
r

o
f

S
p
i
k
e
s
t<0.168s 0.168s <t<0.334s

0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Solution
N
u
m
b
e
r

o
f

S
p
i
k
e
s
t<0.168s 0.168s <t<0.334s

Figure 26 Results of NSGA-II third trial Figure 27 Results of NSGA-II fourth trial

Although the results from each of the four trials vary, they all illustrate that every trial had
solutions where the neuron fired very rapidly in the first half of the firing period and very slowly
in the second half of the firing period. This was the desired result of the experiment. NSGA-II is
thus a valuable tool to aid a trial and error approach to finding parameters to cause neurons to fire
in a specified manner, as although it cannot find parameters to cause the neuron to behave exactly
as desired, it can still find parameters that cause the neuron to behave in the desired manner.
49
The parameters shown in Table 7 result in the neuron firing 16 times in the first 0.168s and once
in the following 0.166s. This result occurred in the fourth trial. They were applied to the half-
centred oscillator shown in Figure 16. The values of I
1-2
and I
2-1
were found through trial and error
and result in oscillations of the oscillator at 1.5Hz. Figure 28 shows the resulting spike trains from
CPG1 and CPG2.

Table 7 parameters found using NSGAII for oscillations at 1.5Hz
Predefined values Found using NSGAII Found by trial and
error
V
thresh
c b a t
reset
(s) g
inc
d tau I
1-2
=I
2-1
0.4 0 50 150 0.2 0.2736 -35 1.6251 -127181

0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
CPG1
CPG2

Figure 28 Half-centred oscillator at 1.5Hz using parameters found through NSGAII and trial and
error

50
11. FILTER AND SCALING DESIGN METHOD

The joint angle trajectories were extracted from the CPG by adding or subtracting the outputs of
the relevant motor neurons and then filtering the resulting spike train. The filtered spike train was
then scaled to lie between the desired angles of walking. An algorithm of the low pass filter used
is shown below.

Figure 29 algorithm for LPF (Low Pass Filter)

Get signal to be filtered and store in signal
i=length(signal)?
Loop for i = 1 to length (signal)
Initialise all variables to 0 unless otherwise specified
lasttime = tspike,
tspike = 0
Manipulate (e.g. scale) or plot filtv as desired
Yes
No
signal(i)=0?
tspike = tspike+0.002
t RC
t
fc
RC
t
+
=
=
=
2
2
1
2
002 . 0
2
2

y x y
x sig x
lasttime B i signal sig
t RC
t
lasttime A
fc
RC
) 1 (
) 1 (
) 1 ( ) (
1
) 1 (
2
1
1
2 2
1 1
1
1

+ =
+ =
=
+
=
+ =

Add y to filtv
No (Spike in Signal)
Yes (No spike in signal)
51
In the above algorithm lasttime is a variable used to store the time between the last two spikes in
the input signal and tspike is a variable used to store the time since the last spike occurred in the
input signal. filtv is a vector used to store the filtered signal and A,B, fc
1
and fc
2
are filter
parameters.

The same method was used to scale both the hip and knee filtered signals. The filtered signals are
scaled as follows:
range of motion = maximum desired angle minimum desired angle
filthigh = absolute value of maximum filtered value
filtlow = absolute value of minimum filtered value

if filtered value 0
scaled angle = minimum desired angle+
2
motion of range
+
2
motion of range
filthigh
value filtered

else if filtered value < 0
scaled angle = minimum desired angle+
2
motion of range
+
2
motion of range
filtlow
value filtered

It is necessary to split the scaling of the filtered values into two sections, one for positive values
and one for negative values, because the filtered wave form may not be symmetric.

The following method was used to generate angle trajectories from an input spike train:
1) Get Power Spectral Density of input signal to see under what frequency (fc
1
) the low
frequency power content lies.
2) Use trial and error to determine values of A, B and fc
2
that result in a smoothly filtered
signal. fc
2
should be set lower than fc
1
.
3) Scale filtered signal to desired range using above method.

52
12. FILTER AND SCALING RESULTS

The values of fc
1
, fc
2
, A and B are specific for each signal to be filtered. The following results
only apply to the spike train used in this example, which oscillates at 1.5Hz. The filtered signal
was scaled such that it ranged between -10 and 35. The spike train was created using a half-
centred oscillator and subtracting the output of the one neuron from the other one. The parameters
used in the oscillator are the same as those given in Table 7

0 0.5 1 1.5 2
-1
-0.5
0
0.5
1
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)

0 50 100 150 200 250
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
frequency (Hz)
P
o
w
e
r

c
o
n
t
e
n
t

(
W
a
t
t
s
)

Figure 30 Input spike train to filter Figure 31 Power Spectral Density of input signal
0 0.5 1 1.5 2
-10
0
10
20
30
40
J
o
i
n
t

A
n
g
l
e

(
D
e
g
)
Time (s)

0.54 0.55 0.56 0.57 0.58 0.59 0.6 0.61 0.62
24.5
25
25.5
26
26.5
27
27.5
28
28.5
J
o
i
n
t

A
n
g
l
e

(
D
e
g
)
Time (s)

1
=15Hz,
fc
2
=1Hz, A = 0 and B = 0
Figure 33 Enlarged view of ripple and sampling
on scaled signal
0 0.5 1 1.5 2
-10
0
10
20
30
40
J
o
i
n
t

A
n
g
l
e

(
D
e
g
)
Time (s)

1
=15Hz,
fc
2
=1Hz, A = 140 and B =15

Sample on ripple
Goes against trend of
signal
20ms
Trend
Ripple
53
Power spectral analysis of the spike train reveals that most of the low frequency signal content
occurs below 15Hz. Thus, fc
1
was set to 15Hz.

Figure 32 shows that if the input signal is filtered with fc1=15Hz, fc2=1Hz and both A and B are
set to 0, then signal there is ripple in the output signal.

This is a problem because the sampling for the PWM signal may sample the scaled signal on the
ripple (Figure 33), causing the effected servo motor to try and reverse its direction for one period.
This may not cause the RedBot to trip, but it will result in very high torques within the servo
motor, as the servo motor tries to reverse its direction of movement for one period before it
continues in its original direction. This is undesirable and leads to unnecessary wear on the servo
motor.

Figure 34 shows that the ripple in the signal can be removed by using the following settings:
fc
2
=1Hz, A=140 and B=15.

54
13.DIFFERENT CPG NETWORK ARCHITECTURES

All neurons in the following networks are of the IFA2 type and the networks were implemented
using CPGFRAMEWORK2. There is at least a 180 phase difference between the firing patterns
of the white coloured and the gold coloured neurons shown in the following networks. For the
purposes of this thesis, the coupling between the left and right legs was not used as the hips only
had to have a 180 phase difference. If, however, the networks are to be used for gaits other than
walking (e.g. running) then the coupling between the legs would be used to ensure the necessary
phase difference between the hips. The coupling between legs is shown in the following diagrams
only for the sake of completeness.

The inhibitory connections between oscillator (CPG) neurons are not labelled on the following
figures. However, these connections are described in the text using symbols such as I
1-2
which is
used to describe the inhibitory coupling from CPG1 to CPG2. The inhibitory connections
between the oscillator neurons and the motor (MN) neurons are entirely feed forward. That is
they are only from the oscillator neurons to the motor neurons. These coupling connections are
shown on the figures and the subscript in the connection name denotes the phase difference the
inhibitory coupling causes between the motor neuron and the oscillator neuron
55
13.1. Canonical Network
The figure below shows the Canonical Network proposed by Lewis et al (2005), with the
necessary summers and filters added, to produce joint angle profiles for biped walking.

Figure 35 Canonical Network with summing nodes and filtering

The parameters found, using NSGA-II and trial and error, to cause oscillations of a CPG at 1.5Hz
were applied to this network. The values of the inhibitory connections to cause phase shifts of 90
were found using trial and error. These parameters are shown in the table below.

Table 8 parameters for Canonical network to generate walking patterns at 1.5Hz
Oscillator Neurons (CPG1, CPG2, CPG3 and CPG4)
V
thresh
c b a t
reset
(s) g
inc
d tau I
1-2
=I
2-1
=I
3-4
=I
4-3

0.4 0 50 150 0.2 0.2736 -35 1.6251 -127181
Motor neurons (MN1 to MN8)
V
thresh
c b a t
reset
(s) g
inc
d tau I
180
I
270

0.4 0 50 150 0.2 0.2736 -35 1.6251 -127181 -510
8

Figure 36 shows the resulting joint angle profiles (the same filter parameters were used to filter
both the hip and knee spike trains. They were fc
1
=15Hz, fc
2
=1Hz, A=140 and B= 10).

To Hip
Scaling
Hip Flexor
CPG2
CPG1
CPG3
CPG4
MN1
MN4
MN2
MN3
MN6
MN7
MN52
Hip Extensor
Knee Extensor
Knee Flexor
Hip Flexor
Hip Extensor
Knee Extensor
Knee Flexor
MN8

LPF
LPF

LPF
LPF
Left Leg Right Leg
-
-
-
-
+
+
+
+
To Hip
Scaling
To Knee
Scaling
To Knee
Scaling
I
270
I
180 I
180
I
270
I
270
I
180
I
270
I
180
56

0 0.5 1 1.5 2
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Left hip
Left knee
Right hip
Right knee

Figure 36 Joint angle trajectories from Canonical network

The stance and swing legs are highlighted for a single period of walking in the above figure.
When a leg is in the stance phase its knee is almost straight for that entire phase and when it is in
the swing phase its knee is bent for the majority of the phase. This relationship ensures that as the
legs pass under the RedBots body the stance leg lifts the RedBot and the swing leg is bent so that
the RedBot does not trip.
13.2. Double Bump Network
The Canonical Network proposed by Lewis et al produces a simplified knee profile. Figure 12
shows that the human knee angle profile contains a double bump, whilst the knee profiles that
result from Lewis et als network contain a single bump. The following network was adapted
from the Canonical Network proposed by Lewis et al (2005) and is able to produce the double
bump knee trajectory. By using the outputs of certain motor neurons to control both the hip and
knee joints the number of motor neurons in the network is reduced from 8 to 6.

Left leg = stance leg
Right leg = swing leg
Right leg = stance leg
Left leg = swing leg
57

Figure 37 Double Bump Network with summing nodes and filtering

Motor neurons MN1 (left hip and knee extensor), MN2 (left hip flexor, left knee extensor), MN4
(right hip and knee extensor) and MN5 (right hip flexor, right knee extensor) work in exactly the
same way as the hip extensor and flexor neurons in Lewis et als Canonical network to generate
the hip angle profiles. Like the Canonical Network, a 90 phase shift is required between the hip
flexor and knee flexor neurons. However, unlike Lewis et als method the hip flexor and extensor
neurons are also used as the knee extensor neurons. Because there are two extensor neurons and
only one flexor neuron it is possible to produce the double bump in the knee profile.

The parameters from Table 8 were applied to the Double Bump Network and the resulting joint
trajectories (using the following filter parameters: fc
1
=15Hz, fc
2
=2Hz, A = 140 and B = 15) are
shown below.
0 0.5 1 1.5 2
0
50
100
150
Time (s)
A
n
g
l
e

(
D
e
g
)
Left Hip
Left Knee
Right Hip
Right Knee

Figure 38 Joint Angle profiles from the Double Bump Network
To Hip
Scaling
CPG2
CPG1
CPG3
CPG4
MN1
MN3
MN2
MN5
MN4
MN6

LPF
LPF

LPF
LPF
Left Leg Right Leg
-
-
-
-
+
+
+
+
To Hip
Scaling
To Knee
Scaling
To Knee
Scaling
+
+
I
180 I
180
I
180
I
180
I
270
I
270
Left leg = stance leg
Right leg = swing leg
Right leg = stance leg
Left leg = swing leg
58

The stance and swing legs are highlighted for a single period of walking in the above figure.
When a leg is in the stance phase its knee is almost straight for that phase and when it is in the
swing phase, its knee bends as the leg swings beneath the body. This relationship ensures that as
the legs pass under the RedBots body the stance leg lifts the RedBot and the swing leg bends so
that the RedBot does not trip.
13.3. Modifications of networks to reduce coupling weights
The inhibition signals necessary to cause 90 phase shifts at 1.5Hz in the above networks are
extremely large. The coupling weights were stored in 32 bit variables when implemented on the
PIC and there was concern that at a different frequency the magnitude of the Inhibition signals
may become so large that the weights can no longer fit in 32 bit variables and 64 bit variables
would have to be used. This is undesirable as it wastes PIC memory and processing speed. The
above two networks were adapted as follows to ensure that the inhibition signals remain within
the 32 bit range.

Figure 39 Canonical Network 2
To Hip
Scaling
CPG2
CPG1
CPG3
CPG4
MN1
MN4
MN2
MN3
MN6
MN7
MN5
MN8

LPF
LPF

LPF
LPF
Left Leg
Right Leg
-
-
-
-
+
+
+
+
To Hip
Scaling
To Knee
Scaling
To Knee
Scaling
I
180 I
180
I
180
I
180
I
180+
I
180+
I
180+
I
180+
I
90
I
90
I
90
I
90
59

Figure 40 Double Bump Network 2

The basic structure of the above two networks and the manner in which the joint angle profiles
are generated is the same as that described previously. The only changes regard the neurons that
are phase shifted by 90, the magnitude of the inhibition signal used to phase shift these neurons
has been reduced by using two inhibition signals as opposed to one. The additional inhibition
signals are shown as blue dashed lines in the above two figures.

The new operation of the network is described through an example. Consider MN3 in the network
shown in Figure 39. A phase shift of 90 between CPG2 and MN3 or 270 between CPG1 and
MN3 is desired. MN3 receives a large inhibition signal from CPG1 causing a phase shift between
CPG1 and MN3 of just more than 180. A second, much smaller inhibition signal is then received
from CPG2 which causes MN3 to only start firing 90 after CPG2 started firing. Once MN3 starts
firing the inhibition signal from CPG2 to MN3 is stopped and MN3 fires in the same manner as if
it only received the single inhibition signal from CPG1. The same principle is applied to all
neurons in the above two networks that need to be phase shifted by 90.

Table 9 parameters to cause 1.5Hz oscillations and 90 shifts in Canonical Network 2 and Double
Bump Network 2
CPG Parameters
v
thresh
c b a t
reset
(s) g
inc
d tau I
1-2
=I
2-1
=I
3-4
=I
4-3

0.4 0 50 150 0.2 0.2736 -35 1.6251 -127181
Motor Neuron Parameters
v
thresh
c b a t
reset
(s) g
inc
d tau I
180
I
180+
I
90

0.4 0 50 150 0.2 0.2736 -35 1.6251 -127181 -157181 -6861
To Hip
Scaling
CPG2
CPG1
CPG3
CPG4
MN1
MN3
MN2
MN5
MN4
MN6

LPF
LPF

LPF
LPF
Left Leg Right Leg
-
-
-
-
+
+
+
+
To Hip
Scaling
To Knee
Scaling
To Knee
Scaling
+
+
-
I
180
I
180
I
180
I
180
I
180+
I
180+
I
90
I
90
60
By splitting the coupling into two, the magnitudes of the inhibition signals necessary to cause 90
phase shifts can safely be realised in 32bits and by scaling I
180,
I
180+
and I
90
they could be realised
in 16bits.

The parameters in Table 9 were applied to the Double Bump Network 2 (the filter parameters
were: fc
1
=15Hz, fc
2
=2Hz, A = 140 and B=15) and the resultant joint angle trajectories are shown
in Figure 41. These joint angle trajectories are the same as the joint angle trajectories shown in
Figure 38, which were generated using the ordinary Double Bump Network.

0 0.5 1 1.5 2
0
50
100
150
Time (s)
A
n
g
l
e

(
D
e
g
)
Left Hip
Left Knee
Right Hip
Right Knee

Figure 41 Joint angle profiles generated from Double Bump Network 2

61
14. C IMPLEMENTATION
14.1. Basic code structure
In order to implement the CPG on the dsPIC30F4011 microcontroller it was necessary to translate
the MATLAB code, with a few modifications to allow for real-time implementation of the CPG,
into C code.

As the Runge-Kutta had a time step of 2ms and the PWM signals used to drive the servo motors
had a period of 20ms, it was necessary to simulate 20ms of CPG activity between PWM updates
in order for the network to run in real time.

Four functions: MAIN, CPG, NEURON and LPF were created to implement the CPG on the
microcontroller.

Function name What it does Corresponding MATLAB
function
MAIN Calls all the initialisation
functions, loops through the code
every 20ms (if possible) and
updates the PWM signals
-
CPG This simulated 20ms worth of
CPG activity, sent the resulting
spike trains to LPF and scaled the
filtered data.
CPGFRAMEWORK2
NEURON Contains the Runge-Kutta
algorithm to update a single
neurons parameters by 2ms
IFA
LPF Filters the joint spike trains LPF

62

Figure 42 MAIN algorithm

Timer 2 counted to 20ms. Once it reached 20ms it reset itself and started counting again. This
was used to ensure that the PWM signals were updated at the correct times.
14.2. Reducing the CPG networks
In order to make the CPG networks as efficient as possible the properties of symmetry were used
to reduce both the Canonical and Double Bump networks to only four neurons. This was made
possible because the hips were 180 out of phase with each other and thus what is the flexor
neuron for one hip can be used as the extensor neuron for the other hip. The same could be done
with the knees. In addition to this the 180 phase shift between the hip neurons allowed them be
used as the half-centred oscillator neurons.

Initialise Timers and
PWM channels
Initialise PWM duty cycles to make
RedBot legs straight
Initialise all CPG algorithm variables
While (0) (i.e. loop for ever)
Call CPG function
Timer 2
finished?
Yes
No
Update PWM signals
63

Figure 43 reduced CPG network architecture

14.3. Implementation of the Double Bump Network on the
RedBot
The Double Bump Network 2, using the reduced network architecture shown in Figure 43, and
the parameters given in Table 9, was implemented on the dsPIC30F4011 to test the joint angle
trajectories on the RedBot. The RedBot successfully walked using the described network and
parameters. A short video of this can be found in Appendix 6.

The walking speed of the RedBot was not at the desired 1.5Hz but instead at 0.75Hz. (This can be
seen by inspection of the video and by looking at the recorded angle sensor data shown in Figure
45). This is as a result of the PIC not being able to process the CPG network in real time. The
time taken to simulate 20ms of CPG code on the PIC was approximately 21ms, however, the
PWM signals could only be updated every 20ms and so the PIC waited for 19ms before updating
the PWM signals and simulating the next 20ms of CPG activity. This results in the CPG running
at half the desired frequency and the PWM signals only being updated every 40ms as opposed to
every 20ms. This explains the slight jerkiness of the RedBot during walking.

The CPG code only requires a small amount of optimisation to run in real time. The easiest way
to do this would be to design a more efficient filter than the one implemented in this thesis (It
appears easier to just change the time step of the Runge-Kutta. However, the affects of the
parameters, found throughout this thesis, on the CPG networks would change as the time step
varied. By changing the filter design, all the parameters found in this thesis would still provide
the same results). Other solutions to this problem are given in the recommendations section.

CPG2
CPG1 CPG3
CPG4
Canonical Network hips only
Double Bump Network hips and knees
Canonical Network knees
Double Bump Network knees
Only present in modified
Canonical and Double
Bump networks
64

Even though the RedBot did not walk at the desired speed, it still walked. This is a very important
result as it illustrates the Double Bump Network architecture is capable of creating joint angle
trajectories for successful biped walking.

Figure 44 RedBot walking at 0.75Hz (1.5Hz desired) using Double Bump 2 Network

Notice that in the first frame the left leg is almost straight, but in the second to fourth frames the
left knee flexes slightly before extending as the left leg passes under the body in the fifth and
sixth frames. This is the double bump knee movement. This can be seen in the captured angle
sensor data shown in Figure 45.

If the video or the above picture sequence is examined closely it is possible to see that when the
right leg is in the swing phase, its foot just skims the ground as it passes under the body. This is
as a result of an incorrect transfer function being used to convert the scaled angle signal to the
necessary PWM duty cycle and as a result the knee joint bent to 140 and not the desired 120.
This can be seen on the captured angle sensor data shown in Figure 45.
65

Figure 45 Joint angle profiles obtained whilst the RedBot was walking

The captured data is similar to although not precisely the same in shape as the simulated angle
profiles shown in Figure 41. The biggest discrepancy, besides the time period, is in the hip
waveforms. Reasons for the difference between the simulated and captured data include:
The right foot of the RedBot skims the ground when it passes under the body in its swing
phase. This results in torques on the servo motors causing them to deviate from their
intended positions. Figure 45 shows that every time the RedBot trips over its right foot
the left hip angle increases slightly before continuing to decline.
The RedBot is taking very large steps. As a result large torques are exerted on the servo
motors as the legs come into contact with the ground. In addition to this gravity exerts
torques on the limbs assisting or impeding their trajectories and the inertias of the
swinging limbs also exert torques on the servo motors as they try to change direction.
These torques cause deviations from the actual position of the servo drive shaft to the
desired position of the servo drive shaft. Although, the servo motors do have feedback to
ensure accurate position control, the control is not instantaneous.
The transfer functions to convert the simulated angles to the necessary PWM signals are
not exact.
There is a certain amount of play on the servo motors, thus when a force is exerted on
the servo motor the angle of the servos drive shaft may change without the servo
controller realising that it is in the incorrect position.

Inspection of the video also reveals that the feet of the RedBot slip whilst walking.
0.5 1 1.5 2 2.5 3 3.5 4
0
50
100
150
200
Time (s)
A
n
g
l
e

(
D
e
g
)
Left Hip
Left Knee
Right Hip
Right Knee
66
15. EVOLUTION OF CPG PARAMETERS METHOD

Note: For all experiments from this point on the value of c was set to 0.
15.1. Automating the search for parameters
Lewis et al (2005) showed that it is possible to create effective joint angle trajectories for walking
by phase shifting the knee flexor and extensor neuron firing patterns by 90 from the hip flexor
and extensor neuron firing patterns In addition to this, by varying the phase shift between the
knee and hip neuron firing patterns, different styles and types of gait can be generated. However,
finding the parameters necessary to cause oscillations of the CPG at a desired frequency with the
necessary phase shifts is a difficult process as the mathematical tools for this have not yet been
developed. Whilst it is possible to find the necessary parameters through a trial and error
approach as shown in Chapter 9, it is a time consuming and tedious process.

The aim of this experiment was to develop an automated method to find the necessary parameters
for a desired frequency of oscillation of the CPG shown below, with a phase shift of p between
CPG1 and CPG3. Although the network used in this experiment is very simple, it is easy to apply
the parameters for it to Lewis et als Canonical Network or the Double Bump network to create
the necessary angle trajectories for walking.

Figure 46 CPG network layout used in the automated search for parameters

De Garis (1990) and Lewis et al (1992) showed that the rate of convergence of the genetic
algorithm can be increased by using the concepts of Behavioural Memory and Staged Evolution.
I
1-2 I
2-1
CPG3
I
2-3
CPG1
CPG2
67
The method used in this thesis to find parameters for the CPG utilises both the above concepts to
increase the speed of convergence of the genetic algorithm used.

Staged evolution was applied by first evolving the parameters necessary to cause oscillations of
the CPG at the desired frequencies. Once this was achieved and the results verified, the
parameters necessary to cause the phase shifts were evolved.

The genetic algorithm FBGAMIN was used to search for all the parameters. FBGAMIN finds
parameters such that the fitness function is minimised. Slight modifications were made to the
function FBGAMIN to ensure that the initial population of the search covered a much larger
parameter space than originally programmed and to stop the algorithm once the fitness reached a
certain value. These modifications are outlined below:
The initial population size of FBGAMIN is 200. Originally the populations gene pool
was generated by assigning each gene a random number between -5 and 0. The
population was then seeded by making a single chromosome equal to the trial solution.
FBGAMIN was modified so that 1 chromosome was the trial solution and the remaining
199 chromosomes were generated by multiplying each gene of the trial solution by a
random number between 0 and 5.
Originally FBGAMIN searched for the desired number of generations regardless of the
fitness of the solution. This was changed so that if the fitness of a solution met a
predetermined criterion the search would end. If the fitness never met the criterion then
the search would continue for the specified number of generations.

The different stages of evolution, the experiments within each stage and how the concept of
Behavioural Memory was applied to each experiment are now described.

68
15.1.1. Stage 1: Evolving oscillator parameters
The first stage of evolution consisted of finding parameters to cause oscillations of CPG1 and
CPG2 at the following frequencies: 0.5Hz, 0.75Hz, 1.00Hz, 1.25Hz, 1.50Hz, 1.75Hz, 2.00Hz,
2.25Hz, 2.50Hz, 2.75Hz, 3.00Hz, 3.25Hz and 3.50Hz. Parameters to cause oscillations at 0.50Hz
were found first.

Once parameters were found to cause oscillations at a particular frequency, the fitness function
was changed to optimise parameters for the next frequency and the previous frequencys solution
was used to seed the initial population used for the search. By changing the fitness function to
optimise parameters for the new frequency and by using the best parameters found for the
previous frequency de Gariss concept of Behavioural Memory is being applied to increase the
speed of the search.

The objective function used was called OSCILLATIONS and it used CPGFRAMEWORK2, with
minor alterations, as its basic algorithm. Figure 47 shows an overview of the process used to find
the parameters, Figure 48 shows an overview of the OSCILLATIONS function and Figure 49
provides a detailed algorithm for OSCILLATIONS.

In OSCILLATIONS t
reset
is linked to the desired frequency of oscillation, f
osc
, via the following
equation
) (
24
11
s
f
t
osc
reset
=
This ensures that if either CPG1 or CPG2 do not fire for
12
11
of half a period then the potassium
concentration of the respective neuron is set to 0.

69

Figure 47 Algorithm for automated parameter search to cause oscillations of CPG1 and CPG2 at
different frequencies

Figure 48 Overview of OSCILLATIONS

CPG1 is allowed to burst for half a period before measurements for the fitness function are taken.
The reason for this is because the objective function, PHASES, which is used to find the phase
shift between CPG1 and CPG3 is based on OSCILLATIONS and this half period initialisation
time makes the basic algorithm of OSCILLATIONS compatible for use in PHASES.
Initialise all variables
f
osc
=3.5Hz?
Loop from f
osc
= 0.5Hz to 3.5Hz in steps of 0.25Hz
Pass trial solution and f
osc
to modified FBGAMIN to find parameters using
OSCILLATIONS
Initialise trial solution
Store best solution, fitness and number of generations taken to find solution
trial solution = best solution
Display desired data e.g. actual frequency of oscillation vs. desired
frequency of oscillation
Yes
No
CPG2
start
CPG1
start

CPG1
end

Time (s)
0 q=0 q=2
Start simulation of neurons
CPG2
start
CPG1
start
CPG1
end
Time (s)
0 q=1
Break from simulation and calculate fitness function
CPG1 Spiking patterns
70

Figure 49 Detailed algorithm of OSCILLATIONS

Loop from 0 to 2/f
osc
in steps of 0.002s
th

th
Is v
i
>vthresh?
v
i
=c
desired values
other neurons to 0
Time period up?
Yes
No
Yes
Yes
No
No
Add 1 to i
th
row of spikev
Add 0 to i
th
row of spikev
Increment i
th
tfire(i) = 0
tfire(i) > t
reset
?
g
i
=0
Yes
Same as initialise CPG in CPGFRAMEWORK2
Frequency of oscillation (f
osc
) passed in from calling function
CPG2
start
= time,
q = 1
i=2?
i=1? q=1?
CPG1
start
= time, q = 2
q=0 q=2
CPG1
end
= time,
Break from loop
Calculate fitness function
Return fitness to calling function
Yes Yes
No
No
No
No
Yes
Connection
Point
Crossover
Point
Yes
Yes
No
No
t
reset
=11/(24f
osc
)
71

The fitness function was calculated as follows:
( ) ) ( 3 2 1
) ( 0 3
) ( 10000 3
0 1
) ( 1 1
2
2
) ( 2 1
2
1
) (
1
2
2
2
2
2
2
2
s error error error fitness
s error
else
s error
CPG if
s CPG CPG
T
error
s CPG CPG
T
error
s
f
T
end
start end
osc
start start
osc
osc
osc
+ + =
=
=
=
+ =
+ =
=

The fitness consists of three components: error1, error2 and error3:
error1 is the square of the time difference between the amount time CPG2 spikes over
and half the period of oscillation. error2 is the square of the time difference between the
amount of time CPG1 spikes over and half the period of oscillation. By splitting the error
responsible for describing the frequency of oscillation into two terms, FBGAMIN
optimises for a 180 phase difference between CPG1 and CPG2.
CPG1
end
is initialised to 0 at the start of the algorithm. If no oscillations occur then the
simulation time period will end and CPG1
end
will still be zero. This is the worst case
result and hence error3 is made large so that the final fitness illustrates this.

The following steps are specific for the results obtained in Chapter 16:
The values shown in Table 10 were used as the trial solution to find parameters to cause
oscillations of CPG1 and CPG2 at 0.5Hz. These values result in CPG1 firing at an initial
rate of 125 spikes per second with no oscillations between CPG1 and CPG2.
The maximum number of generations allowed before the search terminated was set to 10.
The fitness value at which the search terminated at was set to 910
-6
s
2
.

Table 10 Trial solution for 0.5Hz oscillations
v
thresh
a b g
inc
d I
1-2
0.5 100 50 0.02 -60 1.00 -1500
72
15.1.2. Stage 2: Evolving phase shift parameters
The second stage of evolution was conducted in two different ways:
The first way was to find parameters, for every frequency of oscillation, which caused a
phase shift of p between CPG1 and CPG3.
The second way was to find the parameters necessary to cause phase shifts of 15, 30,
45, 60, 75 and 90 between CPG1 and CPG3 for a single frequency of oscillation, f
osc
.

Both the above experiments used the same objective function called PHASES. PHASES uses
OSCILLATIONS, with minor alterations, as its basic algorithm. Figure 50 shows an overview of
the algorithm used to find parameters for a single phase shift p at different frequencies. Figure
51 provides an overview of the algorithm used to find parameters for various phase shifts at a
single frequency. Figure 52 provides an overview of PHASES and Figure 53 provides a detailed
algorithm for PHASES.

Figure 50 Algorithm for automated search for parameters to cause a single phase shift between
CPG1 and CPG3 at different frequencies
f
osc
=3.5Hz?
Loop from f
osc
= 0.5Hz to 3.5Hz in steps of 0.25Hz
Pass trial solution, p, CPGparams and f
osc
to modified FBGAMIN to find
parameters using PHASES
Display desired data- e.g. actual phase shift vs. desired phase shift for each different frequency
Yes
No
Set p to desired phase shift
trial solution = parameters to cause CPG to oscillate at 0.5Hz
Load parameters to cause oscillations at desired frequency into CPGparams
73
The initial trial solution is the parameters to cause oscillations of the CPG at 0.5Hz, however once
the first solution has been found, the fitness function is changed to optimise parameters to cause a
phase shift of p at the next frequency and the best solution for the previous frequency is used as
the trial solution. This points FBGAMIN to the correct region of the parameter space and the
concept of Behavioural Memory is used to increase the speed of convergence of the algorithm.

Figure 51 Algorithm for automation of parameter search to cause different phase shifts between
CPG1 and CPG3 at a single frequency.

By changing the fitness function to optimise parameters for the new phase difference and by
using the best parameters found for the previous phase difference, de Gariss concept of
Behavioural Memory is being applied to increase the speed of the search
p=90 ?
Loop from p = 15 to 90 in steps of 15
Pass trial solution, p, CPGparams and f
osc
to modified FBGAMIN to find
parameters using PHASES
Load parameters to cause oscillations at f
osc
into CPGparams
trial solution = CPGparams
Display desired data- e.g. actual phase shifts vs. desired phase shifts
Yes
No
Set f
osc
to desired frequency of oscillation
74
Figure 52 Overview of PHASES

The reason why CPG1 is allowed to burst for half a period before measurements are taken can be
seen above. CPG3 receives inhibitory coupling from CPG2. Thus, in order for any phase shifting
between CPG1 and CPG3 to take place, CPG2 needs to have fired in the previous half-cycle of
oscillations.

For the same reasons as in OSCILLATIONS, PHASES links t
reset
to the desired frequency of
oscillation, f
osc
, via the following equation
) (
24
11
s
f
t
osc
reset
=

CPG2
start
CPG1
start

CPG1
end

Time (s)
0 q=0 q=2
Start simulation of neurons
CPG2
start
CPG1
start
CPG1
end
Time (s)
0 q=1
CPG2
start
CPG1
start
CPG1
end
Time (s)
0
Break from simulation and calculate fitness
function
CPG3
start

Phase difference ()
75
Figure 53 A detailed algorithm of PHASES
Loop from 0 to 2/f
osc
in steps of 0.002s
th

th
Is v
i
>vthresh?
v
i
=c
desired values
other neurons to 0
Time period up?
Yes
No
Yes
Yes
No
No
Add 1 to i
th
row of spikev
Add 0 to i
th
row of spikev
Increment i
th
tfire(i) = 0
tfire(i) > t
reset
?
g
i
=0
Yes
Same as initialise CPG in CPGFRAMEWORK2
Frequency of oscillation (f
osc
) and desired phase shift p passed in from calling function
CPG2
start
= time,
q = 1
i=2?
i=1? q=1?
CPG1
start
= time,
q = 2
q=0? q=2?
CPG1
end
= time,
Break from loop
Calculate fitness function
Return fitness to calling function
Yes Yes
No
No
No
No
Yes
Connection
Point
Crossover
Point
Yes
Yes
No
i=3? q=1 or 2?
CPG3
start
=time
Yes
Yes
No
No
No
t
reset
=11/(24f
osc
)
76

The fitness function was calculated as follows:
( ) ) ( 2 1
) ( 0 2
) ( 100000 2
1 3
) ( ) ( 1
) ( 360 ) 1 3 (
) (
2 1
1
2
2
2
2 2
Deg error error fitness
Deg error
else
Deg error
CPG CPG if
Deg e actualphas p error
Deg f CPG CPG e actualphas
Hz
CPG CPG
f
start start
start start
start end
+ =
=
=
<
=
=
=

The fitness function consists of two components, error1and error2:
If CPG3
start
is less than CPG1
start
then CPG3 starts firing before CPG1. This is
undesirable and hence error2 is made sufficiently large so that the fitness function
illustrates this
error1 is the square of the difference between the desired phase shift and the actual phase
shift.

The following steps are specific for the results obtained in Chapter 16:
Each experiment was repeated twice. The first experiment with p=90 and p=45. The
second experiment with f
osc
=1Hz and f
osc
=2Hz.
The trial solutions for each of the two experiments were generated as described in the
relevant algorithms, using the results of the first stage of evolution in the search for
parameters.
The maximum number of generations allowed before the search terminated was set to 10.
The minimum fitness required before the search terminated was 1deg
2
. This ensured that
the resulting phase shift was within 1 of the desired phase shift.

77
15.2. Evolution of a controller to simulate human walking
The aim of this experiment was to evolve a controller capable of producing as physiologically
accurate joint angle trajectories, at 1.5Hz, as possible from a given network of neurons (such as
that done by Inada and Ishii (2004)) and then to make the controller as efficient as possible by
removing any redundant neurons within the network.

The targets for the joint angle profiles were the left leg, human joint angle trajectories extracted
from Gaitlab. In order to make the task of finding all the different parameters more manageable
the principle of staged evolution was used. However, as parameters already existed to cause
oscillations of the CPG at 1.5Hz there was no need to find these parameters again.

The evolution of the controller was split into two steps: the first step was to find parameters that
could generate the desired hip angle trajectory and the second step was to find parameters to
generate the desired knee angle trajectory. The same CPG architecture (shown in Figure 54) was
used to evolve both the hip and knee controllers, as once all the necessary parameters were found
it was an easy process to combine the two networks into one larger network that generated both
patterns.

The different parameters that were optimised are shown on Figure 54.

78
Figure 54 Initial neural network layout used to evolve both the hip and knee controllers

Only the tonic inputs and coupling weights for CPG3 to CPG8 were evolved, the remaining
neuron parameters used were the same as those used in CPG1 and CPG2. CPGFRAMEWORK2
(with t
reset
=0.3s) was used to simulate the network and the modified version of FBGAMIN was
used to optimise the parameters. The network was simulated for two periods and the filtered,
scaled output over the second period was compared to a single period of human hip or knee angle
trajectory to calculate the fitness. By comparing the output in the second period of oscillations,
the network had one period to reach steady state. The fitness function calculated the Mean
Squared Error (MSE) between the simulated angle trajectory and the human angle trajectory as
follows:
( )
) (deg
) ( ) (
2
2
n
n sim n real
fitness
n

=
Where real is the real angle trajectory, sim is the simulated angle trajectory and n is the number
of samples in one period. If the fitness fell below 7.5deg
2
then the optimisation ended.

Figure 55, below, shows the method used to remove redundant neurons from the network.
I
1-3
CPG2
CPG1
CPG3
LPF
CPG4
CPG5
CPG6
CPG7
CPG8
Scaling
I
1-4
I
1-5
I
2-6
I
2-7
I
2-8
A
3
A
4
A
5
A
6
A
7
A
8
fc
1,
fc
2
Tonic input, a
3
Tonic input, a
4
Tonic input, a
5
Tonic input, a
6
Tonic input, a
7
Tonic input, a
8
Resultant
joint angle
profile
I=Inhibitory coupling
a= tonic input
A= Neuron weight to summer
f
c
=cut-off frequency of filter
A
8
79

Figure 55 Algorithm to evolve human hip or knee controller

The parameter values to cause oscillations of the network at 1.5Hz were those values discovered
using NSGA-II and trial and error. These parameters can be found in Table 7. The parameters
used for the trial solution were the same for both the hip controller and knee controller evolutions
and can be found in Table 16 in Appendix 5.

Are there any neurons such that
A
i
< 0.1 A
max
?
Delete neurons from network
Use modified FBGAMIN and CPGFRAMEWORK2 to optimise network till
desired level of accuracy (<7.5deg
2
) or the maximum number of generations
Save optimised parameters and new network architecture
No
A
max
= maximum value of A
3-8

Is fitness of best solution
<14deg
2
?
If neurons were deleted was
fitness of previous solution
acceptable?
Use previous network architecture and
solution
No
No
Yes
Yes
Yes
Initialise parameters for CPG1 and CPG2
Initialise those parameters not being optimised in CPG3-CPG8 to the same
values as used in CPG1 and CPG2
80
16. EVOLUTION OF CPG PARAMETER RESULTS

This chapter is divided into five main sections. The first two sections give and discuss the results
for the first two stages of evolution used in automating the search for parameters. The third
section discusses the results of applying the resultant parameters to Lewis et als Canonical
network. The fourth section gives the results of the evolution of a human controller and the fifth
section shows that the human controller can successfully cause the RedBot to walk.
16.1. Automating the parameter search: Stage 1 results
The results of the search are as follows:
0.5 1 1.5 2 2.5 3 3.5
0.5
1
1.5
2
2.5
3
3.5
Desired Frequency of Oscillations (Hz)
A
c
t
u
a
l

F
r
e
q
u
e
n
c
y

o
f

O
s
c
i
l
l
a
t
i
o
n
s

(
H
z
)
Ideal Results
Actual Results
0.5 1 1.5 2 2.5 3 3.5
10
-2
10
0
10
2
10
4
10
6
10
8
Frequency of oscillation (Hz)
A
b
s
o
l
u
t
e

v
a
l
u
e

o
f

p
a
r
a
m
e
t
e
r
a
b
ginc
d
tau
vthresh
I
Figure 56 Actual frequencies of oscillation vs.
desired frequencies of oscillation using
Figure 57 Absolute values of parameters found by
FBGAMIN vs. the frequency of oscillation they
cause
0.5 1 1.5 2 2.5 3 3.5
0
1
2
3
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

Figure 58 Number of generations FBGAMIN required to find the parameters to cause oscillations at
each frequency

81
Figure 56 shows that FBGAMIN successfully found parameters to accurately cause oscillations at
all the desired frequencies. Figure 58 shows that the maximum number of generations required to
find a particular solution was 3. This illustrates that the search method rapidly converges to
solutions that provide accurate results.

Figure 57 shows the absolute values of the different parameters used in the network (exact
parameter values can be found in Table 12, Appendix 2). These values span up to five orders of
magnitude as the frequency of oscillation changes and some values are so large that they can not
easily be physically realised. However, these values are acceptable for use in a software generated
CPG as they can be easily realised in the digital domain.
82
16.2. Automating the parameter search: Stage 2 results
16.2.1. Results of a single phase shift at variable frequencies
0.5 1 1.5 2 2.5 3 3.5
89
89.5
90
90.5
91
P
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

0.5 1 1.5 2 2.5 3 3.5
44
44.5
45
45.5
46
P
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

Figure 59 Actual phase shift vs. desired 90 phase
shift at different frequencies using parameters
found by FBGAMIN
Figure 60 Actual phase shift vs. desired 45 phase
found by FBGAMIN
0.5 1 1.5 2 2.5 3 3.5
10
-2
10
0
10
2
10
4
10
6
10
8
10
10
A
b
s
o
l
u
t
e

v
a
l
u
e

o
f

p
a
r
a
m
e
t
e
r
a
b
ginc
d
tau
vthresh
I

0.5 1 1.5 2 2.5 3 3.5
10
-2
10
0
10
2
10
4
10
6
10
8
10
10
A
b
s
o
l
u
t
e

v
a
l
u
e

o
f

p
a
r
a
m
e
t
e
r
a
b
ginc
d
tau
vthresh
I

FBGAMIN to cause a phase shift of 90 at
different frequencies of oscillation
FBGAMIN to cause a phase shift of 45 at different
frequencies of oscillation
0.5 1 1.5 2 2.5 3 3.5
0
1
2
3
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

0.5 1 1.5 2 2.5 3 3.5
0
1
2
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

Figure 63 Number of generations FBGAMIN
required to find the parameters to cause a phase
shift of 90 at each frequency.
needed to find the parameters to cause a phase
shift of 45 at each frequency.
83
16.2.2. Results of multiple phase shifts at a single frequency
15 30 45 60 75 90
15
30
45
60
75
90
Desired phase shift (Deg)
A
c
t
u
a
l

p
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

15 30 45 60 75 90
15
30
45
60
75
90
A
c
t
u
a
l

p
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

Figure 65 Actual phase shifts vs. desired phase
shifts at oscillations of 1Hz using parameters
found by FBGAMIN
Figure 66 Actual phase shifts vs. desired phase
shifts at oscillations of 2Hz using parameters
found by FBGAMIN
15 30 45 60 75 90
10
-2
10
0
10
2
10
4
10
6
10
8
Phase shift (Deg)
A
b
s
o
l
u
t
e

v
a
l
u
e

o
f

p
a
r
a
m
e
t
e
r
a
b
ginc
d
tau
vthresh
I
15 30 45 60 75 90
10
0
10
2
10
4
10
6
10
8
Phase shift (Deg)
A
b
s
o
l
u
t
e

v
a
l
u
e

o
f

p
a
r
a
m
e
t
e
r
a
b
ginc
d
tau
vthresh
I

Figure 67 Absolute value of parameters found
using FBGAMIN vs. the phase shifts they cause
at 1Hz
Figure 68 Absolute value of parameters found
using FBGAMIN vs. the phase shifts they cause
at 2Hz
15 30 45 60 75 90
0
1
2
3
4
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s
Phase shift (Deg)

10 20 30 40 50 60 70 80 90 100
0
1
2
Phase shift (Deg)
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

required to find the parameters to cause a
required to find the parameters to cause a

84
16.2.3 Discussion of results
Figure 59, Figure 60, Figure 65 and Figure 66 show that parameters were successfully found to
cause all the desired phase shifts, within the specified 1 of accuracy, at the desired frequencies.
This is evidence that the search method is successful and can be used to find parameters to cause
accurate phase shifts at different frequencies.

Figure 63, Figure 64, Figure 69 and Figure 70 show that the average number of generations taken
by FBGAMIN to find a particular solution was two, whilst the maximum number of generations
needed was four. Thus by using staged evolution and behavioural memory the rate of
convergence of the search was very fast.

Figure 61, Figure 62, Figure 67 and Figure 68 show the absolute values of the parameters needed
to cause the phase shifts at the particular frequencies (see Appendix 2). As was the case for
the parameters found to cause oscillations of the CPG at various frequencies, the parameters
vary vastly in magnitude (up to five orders of magnitude) as the frequency or phase shift
changes. Some of these values are so large that they cannot easily be physically realised,
however they are acceptable for use in software CPGs as they can easily be implemented in the
digital domain.

The above experiments were repeated with I
2-3
being multiplied by a factor of 1000, within the
PHASES algorithm, before being applied to the IFA2 equations. The search provided results of
comparable accuracy in a similar number of generations (these results can be found in Appendix
3). This illustrates that the search is robust and is capable of quickly finding the required
parameters even if one of the variables is altered.
16.3. Application of GA results to Canonical Network
The parameters found, by the automated search method, for frequencies of oscillation between
0.5Hz and 3Hz (the RedBot walking speed limits) and for phase shifts of 90 at each frequency
were applied to the Canonical network developed by Lewis et al. The generated joint angle
trajectories are shown below. Only the trajectories for one leg are shown, the trajectories for the
other leg are the same but phase shifted by 180. The filtering parameters used for each frequency
of walking pattern can be found in Appendix 4.
85
0 0.5 1 1.5 2 2.5 3 3.5 4
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Knee
Hip

0 0.5 1 1.5 2
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Knee
Hip

Figure 71 Hip and knee angles for 0.5Hz walking Figure 72 Hip and knee angle for 1Hz walking
0 0.2 0.4 0.6 0.8 1 1.2
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Knee
Hip

0 0.2 0.4 0.6 0.8 1
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Knee
Hip
Figure 73 Hip and knee angles for 1.5Hz walking Figure 74 Hip and knee angles for 2Hz walking
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Knee
Hip

0 0.1 0.2 0.3 0.4 0.5 0.6
0
20
40
60
80
100
120
140
160
180
Time (s)
A
n
g
l
e

(
D
e
g
)
Knee
Hip

Figure 75 Hip and knee angles for 2.5Hz walking Figure 76 Hip and knee angles for 3Hz walking

Each walking pattern is repeated for two periods and it is possible to see that the knee trajectory is
approximately 90 out of phase with the hip trajectory. Closer inspection of the walking patterns
reveals that as the hip passes through 0 (i.e. underneath the body) whilst the leg is in the stance
phase the knee is almost fully straight and when the hip passes through 0 whilst it is in the swing
phase the knee is almost fully retracted. The above timing relationship is unlikely to cause the
RedBot to trip whilst it is walking. Not enough time was available to test the above walking
patterns on the RedBot. However, Lewis et al (2005) have already proven that it is possible to
86
achieve walking with such angle trajectories and inspection of the above trajectories suggests that
they would work.
16.4. Evolution of a human controller result
The following figures show the final, optimised networks to simulate human hip and knee angle
trajectories. The percentage contribution and sign of each neurons input to the summer is also
shown. The parameters that give the best solution for each network can be found in Table 16 in
Appendix 5.

Figure 77 Neuron network to simulate human hip trajectory

Figure 78 Neuron network to simulate human knee trajectory

The final fitness of the hip network is 9.5 deg
2
. By looking at Figure 77 it is possible to see that
CPG3 only contributes 4% to the final output, which is lower than 10% of the maximum
contribution by CPG5. However, if CPG3 is removed from the network the fitness rises above the
allowed 14deg
2
. The final fitness from the knee network is 13.2deg
2
which is just inside the
CPG2
CPG1 CPG3
LPF
CPG6
CPG7
CPG8
Scaling
Resultant
knee angle
trajectory
+
+
-
-
38%
14%
13%
35%
CPG3
LPF
CPG5
CPG8
Scaling
CPG2
CPG1
Resultant
hip angle
trajectory
-
+
+
4%
55%
41%
87
allowed maximum fitness of 14deg
2
and none of the remaining neurons contribute less than 10%
of the maximum contribution of CPG3.

Figure 79 and Figure 80 show the target angle trajectory (human data), the initial angle trajectory
(starting solution) and the final angle trajectory (final solution) for the hip and knee networks.

Figure 81 to Figure 84 (next page) show the individual spike trains of each neuron in the two
networks as well as the weighted sums of the different spike trains.
0.1 0.2 0.3 0.4 0.5 0.6
-20
-10
0
10
20
30
40
Time (s)
H
i
p

A
n
g
l
e

(
D
e
g
)
Human data
Starting solution
Final solution

0.1 0.2 0.3 0.4 0.5 0.6
110
120
130
140
150
160
170
180
190
Time (s)
K
n
e
e

a
n
g
l
e

(
D
e
g
)
Human data
Starting solution
Final solution

Figure 79 Hip simulation results Figure 80 Knee simulation results
88

0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG1 and CPG2 Spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
CPG1
CPG2
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG3 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG5 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG8 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)

Figure 81 Spike trains of the surviving neurons for the hip controller
0 0.2 0.4 0.6 0.8 1 1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)

Figure 82 The weighted sum of CPG3s, CPG5s and CPG8s outputs

89

0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG1 and CPG2 Spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
CPG1
CPG2
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG3 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG6 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG7 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)
0 0.2 0.4 0.6 0.8 1 1.2
0
0.5
1
CPG8 spi ke trai n
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)

Figure 83 Spike trains of surviving neurons for the knee controller
0 0.2 0.4 0.6 0.8 1 1.2
-1.5
-1
-0.5
0
0.5
1
1.5
Time (s)
S
p
i
k
e

v
a
l
u
e

(
V
o
l
t
s
)

Figure 84 The weighted sum of CPG3s, CPG6s, CPG7s and CPG8s outputs

90
16.5. Implementation of the evolved CPG on the RedBot
The hip and knee networks shown in Figure 77 and Figure 78 were combined and extra neurons
were added for control of a second leg. The resulting network was too large to be implemented
on the PIC in real time, so software was written to simulate one period of walking in MATLAB
and then to download the joint angle trajectories to the PIC (Running the network in real time
using MATLAB and then sending the angle trajectories to the RedBot via the PIC was not viable
as the host PC was too slow to run the MATLAB simulation in real time). The PIC then cycled
through the joint angle trajectories and updated the PWM signals as needed. The data from the
angle sensors was then sampled using the ADC on the PIC and sent back to MATLAB.

Figure 85 RedBot walking using evolved controller

The RedBot walked very well using this controller and a video of the walking can be found in
Appendix 6. Inspection of the video footage shows that the feet of the RedBot slip during
walking.

91
0 0.5 1 1.5 2
0
50
100
150
200
Time (s)
A
n
g
l
e

(
D
e
g
)
Left Knee
Right Knee
Left Hip
Right Hip

Figure 86 Recorded hip and knee joint angle profiles during walking

The recorded joint angle trajectories are very similar to the desired ones shown in Figure 79 and
Figure 80. However, the knee angle profiles are much smoother than the simulated ones. This is a
result of the RedBot walking at a high frequency and the knee servo motors not being able to
overcome the inertia of the shins to move the lower leg in the prescribed manner in the available
time. Other reasons for deviations between the recorded hip and knee angles from the desired hip
and knee angles include.
Torques are exerted on the servo motors from the ground reaction forces and gravity.
These torques cause deviations from the actual position of the servo drive shaft to the
desired position of the servo drive shaft. Although, the servo motors do have feedback to
ensure accurate position control, the control is not instantaneous.
The transfer functions to convert the simulated angles to the necessary PWM signals are
not exact.
There is a certain amount of play on the servo motors, thus when a force is exerted on
the servo motor the angle of the servos drive shaft may change without the servo
controller realising that it is in the incorrect position.

Once the human controller had been built and implemented on the RedBot, correspondence was
made by the thesis supervisor, Dr Ralph Etienne-Cummings, to Dr M. A Lewis. Dr Lewis was
one of the first researchers to apply genetic algorithms to the design of Central Pattern Generators
(Lewis et al, 1992) and was part of the research team that came up with the concept of staged
92
evolution. More recently, Dr Lewis was involved in the design of the Canonical Network (Lewis
et al, 2005) discussed in Chapter 13. The video of the RedBot walking using the evolved
controller was sent to Dr Lewis and he gave the following reply

there is a little slipping, but otherwise, I like it!
The thing I am really interested in is adding a vestibular sensor and
balancing.
can your students take that on?
-Tony

93
17.CONCLUSIONS AND RECOMMENDATIONS
17.1. Conclusions
The necessary hardware was designed and built to interface the RedBot to the host PC via the
dsPIC30F4011 microcontroller. In addition to this, a simulation environment was programmed in
MATLAB to aid the development of the walking algorithms. This environment allowed the
author to simulate the walking motion of the RedBot whilst it was being controlled by one of the
designed CPG methods. The simulation, however, did not accurately take into the account the
effects of gravity and the ground reaction force on foot impact.

Two Central Pattern Generators, using different network architectures, were designed and both
caused successful walking of the RedBot robot. The first Central Pattern Generator was adapted
from the Canonical Network proposed by Lewis et al (2005) and provided a more physiologically
correct knee angle profile than the Canonical Network. This network was named the Double
Bump Network and was implemented on the dsPIC30F4011 microcontroller, however, the
dsPIC30F4011 could not process the network in real time which resulted in the RedBot walking
at half its desired speed.

The second Central Pattern Generator was designed using a genetic algorithm to solve for the
networks final architecture and parameters. This network was able to mimic human joint angle
profiles with a high degree of accuracy. The evolved CPG was larger than the Double Bump
Network and as a result there was no attempt to implement it on the dsPIC30F4011. Instead, a
single periods worth of joint angle profiles were simulated using MATLAB and then
downloaded to the PIC for implementation on the RedBot. The RedBot walked very well and in
real time using this controller.

Furthermore, an automated search method was developed, using a genetic algorithm, to find
parameters for the Canonical Network to generate walking patterns for the RedBot over its entire
frequency range (0.5Hz to 3Hz). There was insufficient time to test these results on the RedBot,
however, the resultant joint angle profiles, achieved through simulation of the parameters on the
Canonical Network, suggest that the parameters should allow the RedBot to walk.

94
The results from the automated search method and the evolution of the CPG controller resulted in
a paper entitled Configuring of Spiking Central Pattern Generator Networks for Bipedal
Walking Using Genetic Algorithms being submitted to the International Symposium on Circuits
and Systems(ISCAS) 2007 for publishing. Copies of this paper can be found in Appendix 6 and
Appendix 7.
17.2. Recommendations
The following recommendations have been made for future work with the RedBot and the Central
Pattern Generators developed in this thesis:
The method used to implement the CPG on the dsPIC30F4011 should be changed such
that the CPG can run in real time. This could be achieved in a number of different ways
including:
Using a larger time step in the Runge-Kutta algorithm.
Using a simpler integration method such as Euler Integration.
Designing a simpler digital filter
Instead of digitally filtering the spike trains, the PIC could output the spike trains
using its IO pins. External circuitry could be used to combine, filter and convert
the spike trains to PWM signals.

The feet of the RedBot should be replaced with feet that offer better traction. This would
prevent the slippage displayed when the two walking patterns were implemented on the
RedBot

Ankle joints should be added to the RedBot. These would allow more complicated
walking patterns to be investigated.

The RedBot should be made to walk on a treadmill as opposed to in a circle. This would
allow the RedBot to remain stationary during testing, preventing the RedBots wires from
getting tangled up, and removing the asynchronicity imposed on the RedBots legs by
forcing it to walk in a circle.

Limits should be placed on the parameter range FBGAMIN is allowed to search over.
This would ensure that parameters found could be implemented in hardware based CPGs.

95
The method used to automate the search for parameters should be applied to a hardware
CPG network. This would allow the GA to optimise the network in real time (possibly
even faster) allowing for on-line reconfiguration of the network parameters to adjust for
any desired change in gait.

Integration of a CPG controller with sensory feedback and an If-Then rules based
controller should be investigated. The CPG methods used in this thesis are only effective
on flat surfaces. Any disturbance to the RedBot will cause it fall. The RedBot could use
the CPG controller for steady state walking. However, if any disturbance is detected then
the If-Then rules could be used to handle the disturbance before returning the RedBot to
the CPG controller.

96
18. REFERENCES

1. Arshavsky Y, Gelfand I and Orlovsky G, The cerebellum and control of rhythmic
movements, TINS, vol. 6,pp. 417-422, 1983.

2. Brown T, The intrinsic factors in the act of progression in the mammal, Proceedings of
the Royal Society London Serial B, vol 84, pp 308-319, 1911

3. Bussel B., Roby-Brami A., Neris O. R. and Yakovleff A., Evidence for a spinal stepping
generator in man. Electrophysiological study, Acta Neurobiol. Exp. (Wars) 56 No1 pp
465-468, 1996.

4. Camp J, Powered Passive Dynamic Walking MSc project report, The Sibley School
of Mechanical and Aerospace Engineering, Cornell University, 1997

5. Cohen A. H and Boothe D. L., Sensorimotor interactions during locomotion: principles
derived from biological systems, Autonomous robots, special issue on biomorphic
robots, M. A. Lewis and M. A. Arbib (Eds), Vol. 7, pp225-238, 1999

6. Cohen, A. h, Holmes, P. J. and Rand R. H, The nature of the coupling between
segmental oscillators of the lamprey spinal generator for locomotion: A mathematical
model, J. Math. Biol., vol. 13, pp 345-369, 1982

7. de Garis H., Genetic Programming: Building Nanobrains with Genetically Programmed
Neural Network Modules, Proceedings of the International Joint Conference on Neural
Networks, July 1990

8. Dimitijevic M, Gerasimenko Y. and Pinter M, Evidence for a Spinal Central Pattern
Generator in Humans, Annals of New York Acad. Sci. 800 pp 360-376, 1998.

9. Duysens J. and Van de Crommert H.W.A.A, Neural control of locomotion: Part 1: The
central pattern generator from cats to humans, Gait and Posture 7, pp 131-141,1998.

10. Eliasmith C. and Anderson C. H., Rethinking Central Pattern Generators: A General
Approach,2000

11. Forsberg H., Grillner S. and Rossignol S., Phase dependent reflex during walking in
chronic spinal cats, Brain Research, vol. 85, pp 103-107, 1975.

12. Hata K., Saeki K. and Sekine Y., A pulse-type hardware CPG model for quadruped
locomotion pattern, International Congress Series 1291, pp 157-160, 2006

97
13. Hooper, S. L., Central Pattern Generators, pp1-12, 2000

14. Huang Q, Yokoi K, Kajita S, Kaneko K, Arai H, Koyachi N, and Tanie K, Planning
Walking Patterns For A Biped Robot, 280 IEEE TRANSACTIONS ON ROBOTICS AND
AUTOMATION, VOL. 17, NO. 3, JUNE 2001

15. Inagaki S., Yuasa H., Suzuki T and Arai T, Wave CPG model for autonomous
decentralized multi-legged robot: Gait generation and walking speed control, Robotics
and Autonomous Systems 54, pp118-126, 2006.

16. Inda H. and Ishii K., Bipedal walk using a Central Pattern Generator, International
Congress Series 1269, pp185-188, 2004.

17. Inman V.T., Ralston H.J., and Todd F., Human Walking, page 11 and 34 , Baltimore:
Williams & Wilkins. 1981

18. Izhekevich E. M., Which Model to Use for Cortical Spiking Neurons?, IEEE
Transactions on Neural Networks, vol 15, no5, 2004

19. Kajita S, Kanehiro F, Kaneko K, Fujiwara K, Harada K, Yokoi K and Hirukawa H
Biped Walking Pattern Generation by using Preview Control of Zero-Moment Point
Proceedings of the 2003 IEEE International Conference on Robotics & Automation, 2003

20. Kanniah J, Lwin Z, Kumar D and Fatt N AZMP management scheme for trajectory
control of biped robots using a three mass model 2nd International Conference on
Autonomous Robots and Agents, 2004

21. Lam, P, Walking Algorithm for Small Humanoid, University of Auckland Department
of Computer Science Masters Thesis, 2002

22. Lewis M, Fagg A and Solidum A, Genetic Programming Approach to the Construction
of a Neural Network for Control of a Walking Robot, IEEE International Conference on
Robotics and Automation, pp 1618-2623, 1992

23. Lewis M, Tenore F. and Etienne-Cummings R., CPG Design using Inhibitory
Networks Int. Conf. Robotics and Automation,2005

24. Lewis, M. A and Bekey G. A, Gait Adaptation in a Quadruped Robot, Autonomous
Robots In Press, 2002

25. Marder E and Calabrese R L, Modelling the leech heartbeat elemental oscillator. II.
Exploring the parameter space, Journal of Experimental Biology 151 pp 317-340, 1996

98
26. Marder E., Kopell N. and Sogvardt K., How computation aids in understanding
biological networks, P. Stein, S. Grillner, A. Selverston and D. Stuart (Eds), Neurons,
networks and motor behaviour, 1997

27. Mushahwar V, Collins D and Prochazka A, Spinal Chord Microstimulation Generates
Functional Limb Movements in Chronically Implanted Cats, Experimental Neurology
163 pages 442-42, 2000.

28. Pinto C. M. A and Golubitsky M., Central Pattern Generators for Bipedal
Locomotion,2006

29. Prochazka A, The fuzzy logic of visuomotor control, Can. J. Physiol. Pharmacol 74,
pp 456-462, 1996

30. Prochazka A., The Man-Machine Analogy in Robotics and Neurophysiology, Journal
of Automatic Control, University of Belgrade, vol12:4-8, 2002

31. Rasche C, Douglas R and Mahowald M, Characterization of a pyramidal silicon
neuron, Neuromorphic Systems: Engineering silicon from neurobiology, L. S. Smith and
A. Hamilton, eds, World Scientific, 1st edition, 1998.

32. Simoni M and DeWeerth S, Adaptation in a VLSI model of a neuron. IEEE
Transactions on circuits and systems II: Analog and digital signal processing 46(7), pp
967-970, 1999.

33. Stein PSG, Grillner S, Selverston AI and Stuart DG, Neurons Networks and Behaviour,
Cambridge, MA: MIT Press, 1997

34. Stein R, Peckham P and Popovic D, Neural Prostheses: Replacing Motor Function after
Disease or Disability, Oxford University Press, New York,1992

35. T. McGeer, Passive Dynamic Walking, International Journal of Robotics Research,
pp. 62-82 (1990)

36. Tenore F., Etienne-Cummings R. and Lewis M., Entrainment of Silicon Central Pattern
Generators for Legged Locomotory Control, Proc. of Neural Information Processing
Systems 16, Thrun S, Saul L and Scholkopf B (Eds.), MIT Press, Cambridge, MA, 2004

37. Tenore F., Jacob Vogelstein R., Etienne-Cummings, R. Anthony Lewis M. and Hasler P.,
A Spiking Silicon Central Pattern Generator with Floating Gate Synapses, IEEE
ISCAS, 2005

38. Trew M. and Everett T, Human Movement, An Introductory Text (Third Edition),
Published by Churchill Livingstone, 1998.

99
39. Weisstein, E W. Van der Pol Equation. From MathWorldA Wolfram Web Resource.
http://mathworld.wolfram.com/vanderPolEquation.html

40. Whittle M. Gait Analysis: an introduction (Second Edition), Published by Butterworth
Heineman 1996.

41. Wiley K, Observations on the Evolution of Neural Networks for the Control of a
Simulated Quadruped Robot

100
19. BIBLIOGRAPHY

Greenspan, D, Discrete Numerical Methods in Physics and Engineering, Mathematics
in Science and Engineering, Vol 107
Maron M. J, Numerical Analysis, A Practical Approach. Macmillan Publishing Co.,
Inc. 1982
Morrison N, Introduction to Fourier Analysis, John Wiley and Sons, 1994
Rowland, Todd and Weisstein, Eric W. "Genetic Algorithm." From MathWorld--A
Wolfram Web Resource. http://mathworld.wolfram.com/GeneticAlgorithm.html
Vaughan C. L, Davis B. L and OConnor J. C, Dynamics of Human Gait (Second
Edition), Kiboho Publishers, 1999
Vaughan C.L Gait CD, Kiboho Publishers, 1999

101
20. APPENDICES
20.1. Appendix 1
This appendix contains the parameters to illustrate the principles of tonic spiking and spike
frequency adaptation.

Table 11 showing the neuron parameters to illustrate tonic spiking and spike frequency adaptation
Neuron
parameters
a b d g
inc
v
thresh
c
Tonic Spiking t<0.5s a = 0
0.5st2.5s a = 10
t>2.5s a = 0
50 -25.94 0 1.6251 0.4 0
Spike Frequency Adaptation t<0.5s a = 0
0.5st2.5s a = 10
t>2.5s a = 0
50 -25.94 0.03 1.6251 0.4 0

Note: as there is only one neuron in the network it cannot receive any coupling signal from other
neurons and hence the I term in the IFA model falls away.

102
20.2. Appendix 2
This appendix contains all the parameters discovered using genetic algorithms to cause the CPG
shown in Figure 46 to oscillate at a specified frequency with a specified phase shift between
CPG1 and CPG3. Note: Table 13 only contains parameters for CPG3.

Table 12 parameters to cause oscillations of CPG1 and CPG2 at different frequencies
CPG1 and CPG2 parameter values
Frequency of oscillation (Hz) v
thresh
a b g
inc
d tau I
1-2
= I
2-1
0.5 0.94 156.49 34.09 0.02 -63.27 1.96 -1535
0.75 1.84 262.34 70.06 0.10 -36.10 0.36 -1877
1.00 4.91 1142.84 9.61 0.46 -86.55 0.42 -9300
1.25 18.22 3502.95 6.28 0.92 -81.54 1.85 -27123
1.50 63.24 9213.89 13.07 0.58 -288.28 7.99 -58954
1.75 18.54 13545.59 60.72 0.64 -134.79 16.40 -27554
2 83.58 41454.48 177.11 1.28 -111.89 11.67 -113136
2.25 58.83 61176.68 561.26 0.88 -329.08 24.80 -161002
2.50 64.33 134151.78 379.66 2.59 -408.01 56.83 -489917
2.75 33.67 613114.66 730.04 6.16 -853.70 50.75 -1364622
3.00 23.38 2726918.60 435.42 8.69 -3782.59 80.05 -2834286
3.25 53.56 5463261.60 893.38 4.29 -16416.81 224.67 -9064108
3.5 124.32 13017039.00 867.66 7.28 -14580.48 327.60 -32450941

103
Table 13 parameters to cause 45 and 90 phase shifts between CPG1 and CPG3 at frequencies ranging
from 0.5Hz to 3.5Hz
Frequency of
oscillation (Hz)
Phase
Shift
()
v
thresh
a b g
inc
d I
2-3
45 1.38 380.03 1.84 0.08 -168.15 3.00 -4380
0.50
90 2.17 113.91 3.82 0.07 -140.49 0.43 -5925
45 2.52 297.80 9.17 0.39 -75.80 14.41 -10823
0.75
90 4.80 115.33 11.34 0.27 -308.70 1.16 -24265
45 6.97 1346.92 4.49 0.68 -175.69 63.30 -21324
1.00
90 15.65 566.12 6.86 0.41 -974.08 5.14 -22924
45 9.65 4402.16 8.31 1.99 -745.16 319.79 -93668
1.25
90 87.26 2042.40 8.48 0.40 -1064.60 7.51 -59688
45 13.01 13733.55 9.67 1.09 -3090.52 1225.82 -235840
1.50
90 386.75 7524.14 13.85 1.09 -4876.72 25.56 -184564
45 33.36 22284.72 44.03 3.10 -14271.56 3859.41 -1128085
1.75
90 879.49 34268.29 24.99 3.35 -5522.61 105.70 -938755
45 95.23 18255.65 78.44 1.67 -37168.35 4631.61 -4539466
2.00
90 3640.50 134929.32 24.23 13.84 -9170.55 133.52 -2247035
45 460.12 52766.92 67.20 7.38 -89777.94 8087.86 -2523834
2.25
90 254.55 626074.33 9.42 68.66 -3188.73 360.90 -10505453
45 1382.24 257420.40 49.45 3.20 -386667.90 25219.00 -4920486
2.50
90 1115.40 1420466.20 15.12 312.24 -11788.95 1284.76 -31352885
45 5451.73 757757.84 62.21 13.00 -634973.03 72060.69 -9284768
2.75
90 3180.36 358818.13 61.41 146.52 -6897.72 1466.32 -51984023
45 22406.48 2809909.20 72.36 28.92 -2642579.40 7456.33 -30057675
3.00
90 9829.19 899062.78 55.35 692.82 -4614.65 1556.26 -41126881
45 5001.01 2174062.40 99.98 25.57 -9330833.70 28983.41 -94408769
3.25
90 26363.09 4083356.60 46.36 424.38 -8507.20 6032.57 -110883690
45 12233.27 6882857.70 100.66 51.05 -30684657.00 -12028.83 -260828090
3.5
90 32123.80 18814113.00 100.92 1435.94 -11564.39 15211.58 -289940480
104
20.3. Appendix 3
This appendix contains the results of repeating the experiments for the second stage of evolution
in the search for parameters, where I
2-3
was multiplied by a factor of 1000 within the Phases
algorithm before being applied to the IFA2 equations. The parameters used to make the CPG
oscillate at the different frequencies can be found in Table 12 (Appendix 2).

0.5 1 1.5 2 2.5 3 3.5
89
89.5
90
90.5
91
P
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

0.5 1 1.5 2 2.5 3 3.5
0
1
2
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

Figure 87 Actual phase shift vs. desired 90phase
found by FBGAMIN, with I
2-3
scaled by 1000
Figure 88 Number of generations FBGAMIN needed
to find parameters causing a phase shift of 90 at
each frequency, where I
2-3
was scaled by 1000
0.5 1 1.5 2 2.5 3 3.5
44
44.5
45
45.5
46
P
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

0.5 1 1.5 2 2.5 3 3.5
0
1
2
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

Figure 89 Actual phase shift vs. desired 45phase
found by FBGAMIN, with I
2-3
scaled by 1000
Figure 90 Number of generations FBGAMIN needed
to find parameters causing a phase shift of 45 at each
frequency, where I
2-3
was scaled by 1000

105
15 30 45 60 75 90
15
30
45
60
75
90
A
c
t
u
a
l

p
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

15 60 45 60 75 90
0
1
2
Phase shift (Deg)
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
io
n
s

Figure 91 Actual phase shifts vs. desired phase shifts
at oscillations of 1Hz using parameters found by
FBGAMIN, where I
2-3
was scaled by 1000
required to find the parameters to cause a particular
phase shift at 1Hz, where I
2-3
was scaled by 1000
15 30 45 60 75 90
15
30
45
60
75
90
A
c
t
u
a
l

p
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

15 30 45 60 75 90
0
1
2
3
Phase shift (Deg)
N
u
m
b
e
r

o
f

g
e
n
e
r
a
t
i
o
n
s

Figure 93 Actual phase shifts vs. desired phase shifts
at oscillations of 2Hz using parameters found by
FBGAMIN, where I
2-3
was scaled by 1000
required to find parameters to cause a particular
phase shift at 2Hz, where I
2-3
was scaled by 1000

106
Table 14 parameters to cause phase shifts of 90 and 45 for different frequencies of oscillation
Frequency of
oscillation
(Hz)
Phase
Shift
()
v
thresh
a b g
inc
d I
2-3
45 0.94 156.49 34.09 0.0224 -63.27 1.96 -1535
0.50
90 2.64 781.60 9.95 0.1059 -221 1.89 -1097
45 2.30 675.57 27.22 0.0767 -213.42 4.89 -7204
0.75
90 2.88 1961.18 16.29 0.4072 -471 7.52 -4126
45 0.45 736.78 48.89 0.0819 -946.76 15.18 -34166
1.00
90 5.34 6147.84 19.96 0.7949 -1031 12.59 -10191
45 0.69 1042.30 62.80 0.2833 -2283.42 29.62 -40952
1.25
90 22.46 7682.46 31.89 0.4831 -5074 11.10 -48370
45 2.69 3940.84 75.43 0.9977 -4414.97 102.87 -109282
1.50
90 78.86 23298.16 40.67 0.0076 -13535 17.27 -217336
45 6.38 4046.99 82.62 2.5841 -12046.96 392.11 -74251
1.75
90 70.47 77877.13 58.73 0.0004 -13197 59.02 -801769
45 18.84 4269.54 148.90 0.4557 -29393.73 217.21 -217676
2.00
90 95.69 277129.28 55.65 0.0004 -14656 77.97 -812117
45 27.71 21422.51 124.76 0.2388 -46573.82 639.56 -163563
2.25
90 269.76 407226.95 71.08 0.0014 -16061 279.59 -2939079
45 18.06 45090.12 133.07 0.5465 -35237.90 99.45 -279583
2.50
90 464.10 1265451.70 79.57 0.0685 -62520 1208.29 -9631343
45 76.41 116650.86 152.06 0.0541 -94508.49 200.18 -825321
2.75
90 212.98 4808143.50 80.95 0.1520 -87115 3173.42 -9389411
45 153.48 74153.41 212.50 0.0905 -218076.53 939.84 -1447194
3.00
90 44.32 10237119.00 95.85 0.1343 -400925 4486.16 -38877038
45 448.73 143345.63 300.62 0.2793 -808771.66 4109.64 -6579074
3.25
90 138.09 5338293.90 129.04 0.2326 -1617912 7978.52
-
159846050
45 11.84 127347.56 177.91 1.3883 -618366.66 18844.75 -197560
3.5
90 15.31 490812.77 187.47 3.5513 -1315474.60 35657.81 -740858
107
Note: Table 14 only shows the parameters for CPG3. The parameters to cause CPG1 and CPG2
to oscillate at the specified frequency can be found in Table 12. In order to apply the parameters
from Table 14 to any CPG such as the one shown in Figure 46 it is first necessary to multiply the
value of I
2-3
by 1000.
20.4. Appendix 4
This appendix contains the filter parameters used in filtering the spike trains that emerged when
the parameters found using the automated search method were applied to Lewis et als Canonical
network.

Table 15 Filter parameters used in applying the parameters found by FBGAMIN to Lewis et als
Canonical Network
Frequency of walking (Hz) Joint fc
1
(Hz) fc
2
(Hz) A B
Hip 15 1 160 8 0.5
Knee 15 0.25 160 8
Hip 15 1 160 0 1
Knee 15 2.75 160 0
Hip 15 5 160 0 1.5
Knee 15 5 160 0
Hip 15 4 160 0 2
Knee 15 7 160 0
Hip 15 4 160 0 2.5
Knee 15 7 160 0
Hip 15 8 160 0 3
Knee 15 7 160 0

108
20.5. Appendix 5
This appendix contains the trial and final solutions used in evolving a controller to mimic human
hip and human knee joint angle profiles.

Table 16 Trial and final solutions for the evolution of hip and knee controllers
Parameter Initial solution
(hip and knee)
Final hip solution Final knee solution
a
3
150 1182.65 1185.11
a
4
150 2.53 336.00
a
5
150 1277.39 1461.66
a
6
150 353.88 336.89
a
7
150 777.55 631.80
a
8
150 1222.70 439.02
I
1-3
-127181 -167161.40 -359148.03
I
1-4
-5000000 -19619310.26 -40311472.66
I
1-5
-2000000 -10414907.91 -8082115.75
I
2-6
-127181 -1765546.48 -209697.29
I
2-7
-2000000 -21532531.96 -7241962.09
I
2-8
-5000000 -31094521.07 -12714046.89
A
3
1 2.44 9.45
A
4
-1 -7.64 -3.42
A
5
0.01 2.00 0.08
A
6
1 3.12 3.20
A
7
4 31.04 -0.02
A
8
-1 -22.50 -8.53
fc
1
15 4.19 24.63
fc
2
1 3.17 4.65

The above table shows the final solutions for all the parameter optimised in the network,
including the final parameters for those neurons deleted from the network.

109
20.6. Appendix 6 (See attached CD)
This appendix can be found on the CD attached to this thesis. It contains the videos of the RedBot
walking using the Double Bump Network 2 and the evolved controller network. The CD also
contains a selection of the most important MATLAB M files used during this thesis. A listing and
description of these M files can be found on the CD in the PDF document called M-files. The CD
also contains a PDF of this thesis write up and a PDF of the paper that resulted from this work.
20.7. Appendix 7
This appendix contains a copy of the paper entitled Configuring of Spiking Central Pattern
Generator Networks for Bipedal Walking using Genetic Algorithms that was submitted to the
International Symposium on Circuits and Systems(ISCAS) 2007 for publishing.

The paper can be found on the following page.
Configuring of Spiking Central Pattern Generator Networks for
Bipedal Walking Using Genetic Algorithms
Alex Russell, Garrick Orchard
Department of Electrical Engineering
University of Cape Town
Rondebosch, Cape Town, RSA
{rssale006, orcgar001}@mail.uct.ac.za
Ralph Etienne-Cummings
Department of Electrical and Computer Engineering
Johns Hopkins University
Baltimore, MD, USA
retienne@jhu.edu

AbstractIn limbed animals, spinal neural circuits responsible
for controlling muscular activities during walking are called
Central Pattern Generators (CPG). CPG networks display
oscillatory activities that actuates individual or groups of
muscles in a coordinated fashion so that the limbs of the animal
are flexed and extended at the appropriate time and with the
required velocity for the animal to efficiently traverse various
types of terrain, and to recover from environmental
perturbation. Typically, the CPG networks are constructed
with many neurons, each of which has a number of control
parameters. As the number of muscles increases, it is often
impossible to manually, albeit intelligently, select the network
parameters for a particular movement. Furthermore, it is
virtually impossible to reconfigure the parameters on-line. This
paper describes how Genetic Algorithms (GA) can be used for
on-line (re)configuring of CPG networks for a bipedal robot.
We show that the neuron parameters and connection
weights/network topology of a canonical walking network can
be reconfigured within a few of generations of the GA. The
networks, constructed with Integrate-and-Fire-with-Adaptation
(IFA) neurons, are implemented with a microcontroller and can
be reconfigured to vary walking speed from 0.5Hz to 3.5Hz.
The phase relationship between the hips and knees can be
arbitrarily set (to within 1 degree) and prescribed complex joint
angle profiles are realized. This is a powerful approach to
generating complex muscle synergies for robots with
multiple joints and distributed actuators.

I. INTRODUCTION
Over the past few years there has been a flurry of activity in
developing locomotion controllers for legged robots that mimic
the neural circuits found in limbed and un-limbed organism [1-
3]. In particular, we have witness a number of implementations
of snake-like robots [1,4], quadrupedal and hexapedal robots
[2,5-7,15] and bipedal robots [8-10] that use models of lamprey,
leech, cat, stick insect and other mammalian Central Pattern
Generator (CPG) networks. CPG networks are collections of
neurons that are able to endogenously produce sustained
rhythmic or oscillatory, patterned outputs [11]. The outputs of
central pattern generators are responsible for most rhythmic
motor patterns such as walking, flying, swimming and
breathing. The motivation for all this work is the drive to
develop mobile robotic systems that are equally adept at
handling the terrain complexities and clutter of natural
environments, while using the same minimal amount of energy
as exhibited by living organisms.
Recently, there has also been movement towards interfacing
robots to humans, e.g. prosthetic devices for amputees and
spinal injury patients, which has elevated the need for devices
that seamlessly interface with the human body. For
locomotion and other muscular actuation, as encountered in
reach, grasp and dexterous manipulations, biological CPG
networks are the neural systems with which the robots have to
interact [12]. Hence artificial CPG networks that bridge the gap
between the biological organism and the robot should speak
the same language as their biological counterpart, i.e. they
should be constructed with spiking neurons and process spike
data. The CPG networks described in this paper follows this
philosophy and are constructed with spiking Integrate-and-Fire-
with-Adaptation (IFA) neurons [13].

(a)

(b)
Figure 1: (a) The RedBot biped. The hips and knees are independently
actuated with servo motors. (b) Lewis et als canonical walking network for
RedBot [5]. The central oscillators are used to generate the hip and knee
control waveforms.
Typically, the CPG network for controlling the RedBot biped
in figure 1(a) with hip and knee actuators are constructed with
the canonical walking network shown in figure 1(b) [5]. The
canonical network is composed of 12 neurons, each of which
has 6 parameters. The network itself has 34 connection weights.
Hence, the parameter space (assuming complete independence
of each parameter) has 106 dimensions. Practically, the
dimension of the problem is reduced by allowing similar neuron
types (i.e. central oscillator neurons, left/right hip motor neurons
and left/right knee motor neurons) to have similar properties.
This reduces the dimensionality to 64. This number can be
reduced further if other symmetries are imposed on network
(e.g. connections forming the main oscillator are symmetric and
left and right hips are 180 degrees out of phase and knees are 90
degrees out of phase with hips). Nonetheless, it is clear that the
curse of dimensionality is significant in this network, and it will
only get worse as ankle joints are added, or more distributed
actuators are used per joint [14]. The latter is fundamental in
the prosthetic limb application. Hence, it is necessary to
110
simplify the network, and to find an automated means of
determining these parameters. Furthermore, an approach that
allows on-line modifications as a function of environmental
condition is even more desirable.
This paper describes four main advances in the design of
artificial CPG networks and attempts to address the problems
outlined above: 1) Spiking IFA neurons are used to implement
biologically compatible networks to control a biped with hip and
knee actuators (uses the neural model in [13]); 2) The
canonical walking network in [5] is modified to allow more
general control waveforms for the joints; 3) A hierarchical
Genetic Algorithm (GA) is used to find the parameters of the
network to achieve walking with prescribed frequency and joint-
ankle profiles (this extends the work of Lewis [15] and Inada
and Ishii [10]; 4) Demonstrates GA mitigated on-line
modification of gait patterns and walking speed in the RedBot
biped.
II. THE NEURON, CPG NETWORK AND GENETIC
ALGORITHM
A. The Neuron Model
We chose spiking neurons to construct our networks to stay
somewhat bio-realistic and compatible with biological nervous
systems. Izhekevich in [13] lays out the trade-offs between
implementation complexity and computational power of various
neuron model. For our purpose, the Integrate-and-Fire-with-
Adaptation (IFA) model is computationally efficient whilst
having enough functionality to form our CPG networks. The
IFA model we used includes a slight modification to reset the
adaptation term if the neuron does not fire after prescribe period.
The model for the i
th
neuron in a network of size n is then given
by:
0
) (
) (
,
= >
=
= >
+ + =

g then t t if
g t e
g
c v then v v if
v d g bv a I v
reset fire
i
i
i thresh i
i i i
i n n all
i n i

Where v is the membrane potential, a the tonic input, I
j-I
the
coupling from the j
th
to i
th
neuron, c is the reset voltage, b is the
membrane impedance, g, e, d, are variables for adaptation and
t
fire
is the time since the neuron last fired, t
reset
is the maximum
time the neuron can remain inactive for before the adaptation
term is reset. The value of t
reset
was set to ~50% of the walking
period.
B. The CPG Network
Lewis et al [5] showed that the network of neurons in Figure
1(b), constructed with IFA neurons, can be used to generate
gaits for walking. In that implementation, only specific gaits
(i.e. specific hip to knee relationships), which were particularly
sensitive to the chosen parameters were shown.
We have modified this network to allow arbitrary walking
frequency, gait and joint angle profiles to be realizable and
reconfigurable on-line. To facilitate these three levels of
control, we decompose the canonical network into three parts:
1) the central oscillator to set the walking frequency; 2) the hip
neuron to knee neuron coupling to realize specific gaits; 3)
convergence of the hip and knee neurons onto a network of
motor neurons, each having its own firing characteristics, to
produce a prescribed joint angle profile. Figure 2(a) shows the
network for defining the walking frequency and gait, while 2(b)
shows the convergence network.

(a)

(b)
Figure 2: (a) The simplified caconical CPG network in fig1 1(b) for training
the walking frequency (hip oscillations) and gait (hip to knee relationship).
(b) The convergence network to generate a particular joint angle profile for
the hips. Similar networks are realized for the knees. The circles labelled
CPG # are IFA neurons. Arrows represent excitatory connection and
closed circles are inihibitory connections.
C. The Genetic Algorithm (GA)
Genetic Algorithms allows various combination of the
network topology to be constructed, and their performance
evaluated against desired behavior to determine fitness. The
qualities of the fitter networks are then passed to subsequent
generations of the network. Typically, the convergence rate of
the GA is the stumbling block for this approach. De Garis
(1990) [16] showed that the evolution of a controller can be
sped up by choosing the right starting point and by using two
fitness functions, where the first fitness function is more easily
satisfied and is used to pre-evolve the solution to an appropriate
level. Next the second fitness function is used to further improve
the solution. De Garis named the above concept Behavioral
Memory. This concept was also use by Lewis et al (1992) used
genetic algorithms to optimize the weights in a network of
neurons to control a 12 degree of freedom hexapod robot [15].
To improve the convergence rate of the GA, Lewis used the
concept of staged evolution (which is similar to Behavioral
Memory) which involves breaking the problem into a set of
manageable challenges, which are solved hierarchically. More
recently, Inada and Ishii (2004) also used a genetic algorithm
and a staged evolution approach to optimize parameters for CPG
control of ankle, knee, hip and pitching of the waste movements
in a simulated biped [10]. Non-spiking CPG networks were
evolved to match the desired joint angles in this simulated
system.
We also follow a staged or hierarchical GA process where we
first evolve the desired walking frequency, then the gait and
lastly the joint angle profile. Our network is constructed entirely
with spiking IFA neurons. The genetic algorithm (GA) used
was based on FBGAMIN, a single goal optimizer, which falls
into the class of an Adaptive Mutation Breeder Algorithm
(AMBA) [17]. Breeder algorithms are well suited for the
optimization of real-valued functions and the adaptive mutation
characteristic of the algorithm utilizes a simple feedback loop
within the algorithm to ensure that the mutation rate is always at
111
its optimum. FBGAMIN uses elitism to ensure that the best
solution always propagates, unchanged, to the next generation
and hence the fitness over generations can only improve or
remain the same.
The population size of the GA was set to 200, where 1
chromosome was the suggested trial solution and the remaining
199 chromosomes were generated by multiplying each gene of
the trial solution by a random number between 0 and 5. This
ensured that the initial population searched a very wide
parameter space.
Our fitness function for frequency of walking is:
( ) 3 2 1 ) (
0 3 10000 3 0 1
) 1 ( _
2
2
) 2 , 1 (
2
1
1
2
2
error error error frequency fitness
error else error CPG if
CPG width pulse
T
error
CPG CPG delay
T
error
frequency desired
T
end
+ + =
= = =
=
=

Our fitness function for gait is:
( ) 2 1 ) (
0 2 100000 2 ) 1 ( _ ) 3 ( _
) _ ( 1
360 ) 1 , 3 (
) 2 ( _ ) 1 ( _
1
2
error error gait fitness
error else error CPG set on CPG set on if
e actualphas phase desired error
f CPG CPG delay e actualphas
CPG width pulse CPG width pulse
f
+ =
= = <
=
=
+
=

In both cases, the maximum number of generations the
algorithm was allowed to evolve for was set to 10, however if
convergence was terminated if the fitness went below 910
-6
.s
2

and 1deg
2
, respectively.
Our fitness function for joint angle profile is the Mean
Squared Error (MSE) for one period of the network joint angle
profile and one period of the desired (human) walking data:
n
angle CPG angle desired
fitness
n

=
2
) _ _ (

where n =67. For both knee and hip angles, the maximum
number of generations allowed was 20, however if the fitness
went below 7.5deg
2
then the search ended.
The initial conditions for the IFA neurons are listed below,
which causes CPG1 to fire at a rate of 125 spikes per second
with no oscillations by CPG2 and CPG3.

Initial Conditions for the Neuron Parameters
Vthresh (v) a b e/tau d I
1-2

0.5 100 50 0.02 -60 1.0 -1500
III. RESULTS
Evolving the Walking Frequency: The GA easily and quickly
found the parameters to realize walking frequencies from 0.5Hz
to 3.5Hz (figure 3). The system converged within 3 generations
(not shown). Figure 4 shows the parameters for the IFA neurons
used to construct the master oscillator (CPG1 and CPG2).
Despite the wide range of parameter values, all play important
roles in the network because of the non-linear and off-on nature
of spiking networks. Notice that both IFA neurons have the
same parameters, which reduces the dimension of the problem.
Evolving the Walking Gait: Given the frequency of walking,
we now evolve the hip to knee relationship for effective
0.5 1 1.5 2 2.5 3 3.5
0.5
1
1.5
2
2.5
3
3.5
Desired Frequency of Oscillations (Hz)
A
c
t
u
a
l

F
r
e
q
u
e
n
c
y

o
f

O
s
c
i
l
l
a
t
i
o
n
s

(
H
z
)
Ideal Results
Actual Results

Figure 3: The walking frequency produced by the CPG network
after a maximum of 3 generations.
0.5 1 1.5 2 2.5 3 3.5
10
-2
10
0
10
2
10
4
10
6
10
8
A
b
s
o
l
u
t
e

v
a
l
u
e

o
f

p
a
r
a
m
e
t
e
r
a
b
e/tau
d
tau
vthresh
I

Figure 4: Shows the parameter value distribution for each frequency of
walking. The actual values are scaled based on physical constraints.
stepping. In this network, there is only a feed-forward
relationship from the hip to the knee, hence the parameter search
is simple and converges very quickly, usually after a maximum
of 2 generations to < 1 degree accuracy. Figure 5 shows a plot
of the desired versus actual phase difference between the hip
and knee for a walking frequency of 1Hz. Similar accuracy is
obtained at all the walking frequencies we test (i.e. 0.5Hz to
3.5Hz in steps of 0.25Hz). Figure 6 shows the CPG networks
hip to knee phase distribution when evolved to reach 45 degrees
at various walking frequencies. Again, similar plots are obtained
for different desired gaits with similar accuracy.
15 30 45 60 75 90
15
30
45
60
75
90
A
c
t
u
a
l

p
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

Figure 5: Shows the evolved CPG networks hip to knee phase for a
walking frequency of 1Hz.
Evolving Joint Angle Profiles: Once the frequency and gait
networks have been evolved, the control signals are applied to
the robot. Since the robot are actuated with servo-motors, a
lowpass filtered version of the spikes is generated by the micro-
controller, and PWM signals are sent to the servo-motors. With
no further training, the network displays the walking gait whose
112
joint angle profiles are shown in figure 7(a). This profile is
sufficient to make the robot walk effectively.
0.5 1 1.5 2 2.5 3 3.5
44
44.5
45
45.5
46
P
h
a
s
e

s
h
i
f
t

(
D
e
g
)
Desired phase shift
Actual phase shift

Figure 6: Shows the evolved hip to knee phase (desired phase is 45 degrees)
at different walking frequencies.
As a further demonstration of the power of this approach, we
wanted to impose a more physiological joint angle profile on the
robot. Hence, we evolved a network whose outputs mimic the
shape of the human joint angles [18]. Figure 7(b) shows the
pre- and post-evolution joint angle profiles for one hip and knee,
showing that network has evolved a fairly close replication of
the human joint profiles.

(a)

(b)
Figure 7: (a) Shows the joint angle profile for hip and knee after frequency
and gait evolution only. (b) Shows the joint angle profile after the CPG
network have been evolved to mimic human hip (left) and knee (right) angle
profiles. Black: human, red: pre-evolution and blue:post-evolution.
Making RedBot Walk: In this final step, we hosted the
evolved CPG networks onto our biped and recorded the joint
angles as our robot walked across a ground at 1.5Hz. Figure 8
shows the joint angle profiles for the 2 hips and 2 knees of
RedBot. The robot replicating the human joint angle profile.
With the ability to reconfigure the network within a handful of
generations, the decoupling of the walking frequency, gait and
joint angle profiles (joint angle profile evolution is the most
complex and least likely to be changed on-line) and using
hardware CPG networks as in [5,8,9], we envision that the
system can be easily modified in real-time and on-line.

Figure 8: (a) Shows the recorded joint angle profile for hip and knee after
frequency and gait evolution only for 1Hz walking. (b) Shows the recorded
joint angle profile after the CPG network have been evolved to mimic human
hip and knee angle profiles for 1.5Hz walking.
IV. CONCLUSION
We have developed biologically inspired locomotion control
networks to generate the muscle synergies for walking in a
bipedal robot. The network is constructed with spiking IFA
neurons, and a hierarchical GA is used to determine the neuron
and network parameters for walking. We show that the
frequency of walking evolves quickly and independently of the
gait and the desired joint angle profiles. By first evolving the
frequency, then the gait (hip to knee phase) and then the joint
angle profiles, we are able to obtain stable, low error walking
solutions within a few generations, provided reasonable initial
conditions are used. Furthermore, using our hardware CPG
networks in [5,8,9], to replace the microcontroller neurons, we
expect real-time end-to-end performance (currently the network
update during GA computation prevents real-time evolution).
We finally demonstrated the walking efficacy of our evolved
networks by hosting them in our Redbot bipedal robot.
ACKNOWLEDGMENT
This work is supported by the U. Cape Towns African and
Fulbright Fellowship. We would like to thank Prof. J. Green for
help with genetic algorithms and Dr K. Vaughan for help in
obtaining human walking angle trajectories.
REFERENCES
[1] G. Cymbalyuk et al., Modeling Alternation to Synchrony with
Inhibitory Coupling, Neural Computation, Vol. 12, 2000.
[2] P. Arena et al., A CNN-based Chip for Robot Locomotion Control,
IEEE Trans. Circuits and Systems I, Vol. 52, No. 9, 2005.
[3] K. Nakada et al., An Analog CMOS Central Pattern Generator for
Interlimb Coordination in Quadruped Locomotion, IEEE Trans.
Neural Networks, Vol. 14, No. 5, 2003.
[4] L. Zhenli et al., Serpentine Locomotion of a Snake-like Robot
Contolled by Cyclic inhibitory CPG Model, Proc. of IROS 2005,
Edmonton, Albert, Canada, July 2005.
[5] A. Lewis et al., CPG Design using Inhibitory Networks, Proc. ICRA
2005, Barcelona, Spain, 2005.
[6] H. Chiel et al., Evolution and Analysis of Model CPGs for Walking:
Dynamical Modules, J. Comp. Neuroscience, Vol. 7, No. 99, 1999.
[7] S. Still et al., Four-legged Walking Gait Control Using a
Neuromorphic Chip Interfaced to a Support Vector Learning
Algorithm, Proc. NIPS, Denver, CO, 2000.
[8] M. A. Lewis et al., An In Silico Central Pattern Generator,
Biological Cybernetics, Vol. 88, No. 2, February 2003
[9] F. Tenore et al., Entrainment of Silicon Central Pattern Generators for
Legged Locomotory Control, Proc. of Neural Information Processing
Systems 16, S. Thrun, L. Saul and B. Scholkopf (Eds.), MIT Press,
Cambridge, MA, 2004.
[10] H.Inada and K.Ishii, A Bipedal walk using a Central Pattern
Generator, Brain-Inspired IT I, Elsevier, 2004.
[11] Stein PSG, Grillner S, Selverston AI and Stuart DG, Neurons
Networks and Behaviour, Cambridge, MA: MIT Press, 1997
[12] S. Schaal, et al., Rhythmic Arm Movement is not Discrete, Nature
Neuroscience, Vol. 7, No 10, 2004
[13] E. Izhikevich, Which Neuron Model to Use? IEEE Trans. Neural
Networks, Vol. 15, No. 5, 2004
[14] E. Marder and R. Calabrese, Modelling the leech heartbeat elemental
oscillator. II. Exploring the parameter space, Journal of Experimental
Biology, Vol. 151, 1996
[15] M.A. Lewis et al., Genetic Programming Approach to the
Construction of a Neural Network for Control of a Walking Robot,
Proc. ICRA 1992, Nice, France, 1992.
[16] H. de Garis, Genetic Programming: Modular Neural Evolution for
Darwin Machines, Proc. IJCNN 1990, Washington, DC, Jan 1990.
[17] D. Goldberg, Genetic Algorithms in Search Optimization and Machine
Learning, Addison-Wesley, 1989
[18] C. Vaughan et al., Dynamics of Human Gait, Human Kinetics Press,
Champaign, IL, 1992
0.1 0.2 0.3 0.4 0.5 0.6
110
120
130
140
150
160
170
180
190
Time (s)
K
n
e
e

a
n
g
le

(
D
e
g
)
0.1 0.2 0.3 0.4 0.5 0.6
-20
-10
0
10
20
30
40
Time (s)
H
ip

A
n
g
le

(
D
e
g
)
113

Design and Implementation of A Central Pattern Generator (CP

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Design and Implementation of A Central Pattern Generator (CP

Uploaded by

Copyright:

Available Formats

Design and Implementation of a Central

Pattern Generator (CPG) Control System

You might also like