Professional Documents
Culture Documents
method
Xianfeng Huang
College of Water Conservancy and Hydropower Engineering
Hohai University
xfhuang@hhu.edu.cn, 18539123@qq.com
Outline
Why & When to Use GA?
What is GA?
GA operators
Examples of GA operators
GA parameters
Numerical Example
Applications in Environmental Systems
Summary
Optional Exercise
WHY USE GA?
advantages, disadvantages, heuristic methods
Motivation
We have in our suitcase
Linear Programming (LP),
Non Linear Programming
(NLP), Dynamic
Programming (DP), and
others.
s.t.
0 x1 ,..., xn 5
Generate a sample between 0 and 5.
How ?
Evaluate the generated sample
Save the best values
Selection
(offspring have some characteristics from each
parent)
Some of these offspring may survive to produce
offspring of their ownsome wont
The better adapted offspring are more likely to
survive
Genetic Algorithms
Basic idea
The basic idea is to simulate the natural
evolution mechanisms of Organisms. In a competitive
environment the fittest survive.
s.t.
0 x1 ,..., xn 5
X-vector of decision variables chromosome
(individual)
Crossover
Combine pieces of good parents
A random shuffle on chromosomes
Produce new, possibly better children
(chromosomes)
Requires genetic diversity among the
parents to ensure sufficiently varied
offspring
GA Operators
Mutation
Mutation is a random change of a
chromosome
Change a gene( or some genes),
small movement in the neighborhood
Helpful to prevent premature
convergence
Creates the diversity required by the
Crossover
GA Operators
Summary
Selection involves the process of choosing
chromosomes from the current population for
reproduction according to their fitness values
Crossover involves partial exchange of
information between pairs of chromosomes
Mutation involves a random change in the
chromosomes
GA Operators
Rules of the Game
Chromosomes from the current population need to be
selected for reproduction.
The selection procedure picks out two parent
(chromosomes) at a time, based on their fitness values,
for reproduction.
The two parents then undergo crossover and mutation
operators to produce two offsprings for the new
population.
This selection crossover - mutation cycle is repeated
until a new population is generated.
The higher the fitness value of a chromosome is, so is
its probability to be selected for reproduction.
GA Operators
Put Together to form GA
GA EXAMPLE
Example 1
Example [min f(x)]
2.5
1.5
0.5
f=0.37 at x
= 809
0
0 100 200 300 400 500 600 700 800 900 1000
First Population
2.2
1.8
1.6
1.4
1.2
0.8
0.6
0.4
Generation 1
0.2
0 200 400 600 800 1000 1200
Individual Best
s individu
al
Solution process
2.2
1.8
1.6
0.8
al individu 0.6
al 0.5
0.4
0.3
0 10 20 30 40 50 60 70 80
Generations
Generation
s
EXAMPLES OF GA
OPERATORS
Selection (Tournament, Roulette)
Crossover (One point, Uniform, Blending)
Mutation (Flip bit, Random)
Selection Mechanisms
Roulette (Proportional): Cost/Rank
Within this selection method the likelihood of a
chromosome to be selected is computed according to
its relative fitness (cost) (also known as Proportional
Selection) / or according to its rank
Tournament
Randomly, using a uniform distribution, select
chromosomes of the population (two or more)
Run a "tournament" between the chosen chromosomes
where the relative fitness of each chromosome acts as
its probability to win the "tournament"
Selection Mechanisms
(Roulette/Proportional)
Example
N pairs in the
Present mating pool
population
1
1 Roulette: Pair 1
4
2 Six times
3
3 Pair 2
5
4
5 2
Pair 3
6 2
2N = 6 N=3
Crossover Mechanisms
With a predefined
N pairs in the mating pool Crossover Probability (Pc),
each of the pairs
1 undergoes Crossover (i.e.,
Pair 1 there is a probability of 1
4 - Pc that a specific pair
will move to the next
3
Pair 2 generation unchanged(.
5
Not all the pairs thus
2 undergo Crossover.
Pair 3 In fact in most GA
2
procedures the best
solution at a given
N=3 generation is moved
unchanged to the next
Crossover Mechanisms
(One Point Crossover)
One point: A random crossover point is selected.
The two second parts of the two parents are
switched producing two new offspring strings
For example: The first gene of Offspring #1 with b randomly selected as 0.76
will be: Gene1 (Offspring #1) = 2.34 0.76 (2.34 5.65) = 4.86
Using Blending (a linear combination of the two parents for each gene):
Genei (Offspring #1) = Genei (Parent1) b [Genei (Parent1) Genei (Parent2)]
Genei (Offspring #2) = Genei (Parent2) + b [Genei (Parent1) Genei (Parent2)]
where b is randomly selected between 0 and 1, and each of the genes is rounded
to the nearest integer number
For example: The first gene of Offspring #1 with b randomly selected as 0.62
will be: Gene1 (Offspring #1) = 10 0.62 (10 13) = 11.86 ~ 12
TOO SMALL:
Premature convergence; stuck at a local
optimum (lack of exploration)
Genetic Algorithms
Parameters
Probability of Crossover Pc:
TOO BIG:
Disrupting good solutions before they have the
chance to take hold in the population
TOO SMALL:
Not sufficient mixing to find new (better)
solutions
Genetic Algorithms
Parameters
Probability of Mutation Pm:
1 1
Pm max ,
N L
TOO BIG:
Loss of good solutions Population
size
String
length
TOO SMALL:
Genetic drift (most of the population is to
similar to each other)
Genetic Algorithms
Parameters
Stopping condition:
Obviously the algorithm is required to stop when a
global optimum solution is reached. The problem is
that global (or local) optimal solutions have no labels,
and since no optimality conditions are checked for
stopping, different criteria exist for that:
1. Maximum number of generations.
2. No improvements of the objective function
(fitness) values for several generations.
3. Maximum computational time (budget).
HOW DOES GA WORK?
Example 3
GA Example
Objective is to fitness x 2
maximize
Decision Variables x
0 x 31
Constraints
Encoding scheme Binary base 2
Population size 4
Selection method Proportional
Crossover one point crossover
Pc = 0.9
Mutation Flip one mutation
Pm = 0.001
GA Example
How long the chromosome needs
to be?
Base 2 1 20 1 1
121 1 20 3 1,1
0 x 31 1 22 1 21
1 20 7 1,1,1
1 23 1 22 1 21 1 20 15 1,1,1,1
1 24 1 23
1 22 1 21 1 20 31 1,1,1,1,1
1 25 1 24
1 23 1 2
2
1 21 1
20 63 1,1,1,1,1,1
e.g.
0110 0 24 1 23
1 22 0 21 1 20 0 8 4 0 1 13
1
GA Example
Stage 1: Initial
Population
0 24 1 23
1 22 0 21 1 20 13 169
1170
0.144
1
0.691
0
0.636
Generate
4 random
numbers Strin Binary x Fitnes Proportion cumul
g Number s al ative
s = x2 Selection
0.11 1 01101 1 169 0.144 0.14
0.53 3 4
0.64 2 11000 2 576 0.492 0.63
0.89 4 6
3
Strings 1,01000 8
2, 2, 4 are selected64 0.055
for crossover 0.69
GA Example
Stage 3: Crossover
Generate 1 random number 0.8
0.8 < (Pc = 0.9) perform crossover
01101 01100
11000 11001
Randomly
select a split
point for
each pair 01000 01011
10011 10000
Max 576
729
GA
fitness
Avg.
Example
01100
Stage 4: Mutation
fitness 11001
Generate 1 random number 0.3 01011
0.3 > (Pm = 0.001) NO mutation 10000
Comparison Initial Generation 1st Generation
of maximum Binary x Fitnes Binary x Fitnes
and average Number s Number s
fitness s = x2 s = x2
values from 01101 1 169 01100 1 144
initial 3 2
generation 11000 2 576 11001 2 625
to the first 4 5
Recall
one
Global 01000 8 64 01011 2 729
optimum 10011 1 361 7
GA Example
2 Generation
nd
Coding GA
Optional Exercise
Select an optimization problem of your
choice, for example the Linear
Programming simple problem solved in
class (or any other simple problem for
which you know the optimal solution) and
code yourself a genetic algorithm for its
solution.
No need to submit this exercise.
How To Convert from
Binary Decimal
Binary Evaluate 24 23 22 21 20
Decimal Value 16 8 4 2 1
How To Convert from
Binary Decimal
Binary Evaluate 24 23 22 21 20
Decimal Value 16 8 4 2 1
Maximize
120
100
80
60
40
20
0
0 2 4 6 8 10 12 14 16 18 20
Toy Example
Maximize
Individu f(x)/f(x
al x string f(x) )
0.18571
1 12 1100 104 4
0.19642
2 9 1001 110 9
0.09642
3 2 10 54 9
0.16071
4 14 1110 90 4
0.18571
Crossover
Select candidates based on
proportional (roulette) probabilities:
14
7
12
6
6
9
Crossover
Select random location for crossover,
cross chromosomes, and obtain 2nd
generation
14 1110 1111 15
1 pair
st
7 0111 0110 6
12 1100 1110 14
2nd
pair
6 0110 0100 4
6 0110 0101 5
3 pair
rd
9 1001 1010 10
2 nd
Generation
Individu f(x)/f(x
al x string f(x) )
1 15 1111 80 0.145985
2 6 110 98 0.178832
3 14 1110 90 0.164234
4 4 100 80 0.145985
5 5 101 90 0.164234
6 10 1010 110 0.20073
f(x) = 548
Crossover
Select candidates based on
proportional (roulette) probabilities:
15
4
5
5
14
5
Crossover
Select random location for crossover,
cross chromosomes, and obtain 3rd
generation
15 1111 1100 7
1 pair
st
4 0100 0111 12
5 0101 0101 5
2nd
pair
5 0101 0101 5
14 1110 1101 13
3 pair
rd
5 0101 0110 6
3 Generation
rd
Individu f(x)/f(x
al x string f(x) )
1 7 111 104 0.189781
2 12 1100 104 0.189781
3 5 101 90 0.164234
4 5 101 90 0.164234
5 13 1101 98 0.178832
6 6 110 98 0.178832
f(x) = 584
A Water Resources Example
in Excel and Matlab
10/1/2013
Problem Formulation
TOO BIG:
More computational time to converge
TOO SMALL:
Premature convergence; stuck at a local
optimum (lack of exploration)
Population Too Small
Population = 6
x1 = 2.2840
x2 = 2.7731
X3 = 2.8945
e m a tu re
Pr fval =
n ve rg enc
Co -35.3310
e
time = 1.6 sec
Population Too Large
Population = 1000
>1 hour!
Run aborted by user
Genetic Algorithms
Parameters
Probability of Crossover Pc:
TOO BIG:
Disrupting good solutions before they have the
chance to take hold in the population
TOO SMALL:
Not sufficient mixing to find new (better)
solutions
Crossover Probability Too
Big
Pc = 1.0
N = 10
r u p tion Gen =
D i s
of good 10
o l uti on s
s x1 = 0.7941
x2 = 0.3895
X3 = 5.7700
fval =
-36.1467
Crossover Probability Too
Small
Pc = 0.0
N = 10
Not Gen =
nt 10
sufficie
mixing
x1 = 0.3952
x2 = 2.5000
X3 = 5.0006
fval =
-37.8419
Genetic Algorithms
Parameters
Probability of Mutation Pm:
TOO BIG:
Loss of good solutions
TOO SMALL:
Genetic drift (most of the population is too
similar to each other)
Mutation Probability Too Big
Pm =
0.9
N = 10
Gen =
x110
= 1.6155
Disruption x2 = 1.8158
of good X3 = 0.7650
solutions
fval =
-20.6758
Mutation Probability Too
Small
Pm =
0.0
N = 10
o
Children to Gen =
m i l a r to e ach
si
other x110
= 2.3382
x2 = 0.4413
X3 = 5.2205
fval =
-39.4962
Genetic Algorithms
Parameters
Stopping condition:
e m a tu re
Pr fval =
n ve rg enc
Co -35.3310
e
time = 1.6 sec
Stopping Criteria
Tolerance of change in fitness function
decreased.
x1 = 0.7426
x2 = 0.6488
X3 = 6.3431
fval =
-38.4415