Professional Documents
Culture Documents
Multiobjective Optimization
Eckart Zitzler
Computer Engineering and Networks Lab
Swiss Federal Institute of Technology (ETH) Zurich
Computer Engineering
and Networks Laboratory
Overview
Eckart Zitzler
ETH Zrich
The Challenge:
Standard interface for search algorithms
The End:
Conclusions and outlook
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
Pareto front
Pareto front approximation
x2 decision
space
y2
objective
space
x1
search
MOMH 2002
Tutorial on EMO
y1
evaluation
Eckart Zitzler
ETH Zrich
y2
MOMH 2002
Tutorial on EMO
Pareto optimality:
defines set of optimal trade-offs
(all objectives equally important)
Decision making:
choose best compromise
(based on preference information)
y1
Eckart Zitzler
ETH Zrich
memory
selection
MOMH 2002
Tutorial on EMO
variation
mating
selection
environmental
selection
EA
both
1
1
N:M
randomized
TS
no mating selection
1
1
1:M
deterministic
SA
no mating selection
1
1
1:M
randomized
ACO
neither
1
1
1:1
randomized
Eckart Zitzler
ETH Zrich
0111 fitness = 19
fitness
evaluation
environmental
selection
mating selection
0011
0100
0000
1011
mutation
0011
recombination
MOMH 2002
Tutorial on EMO
Overview
Eckart Zitzler
ETH Zrich
The Challenge:
Standard interface for search algorithms
The End:
Conclusions and outlook
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
y2
Issues in EMO
Diversity
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
aggregation-based
weighted sum
criterion-based
VEGA
y2
parameter-oriented
scaling-dependent
dominance-based
SPEA2
y2
y1
MOMH 2002
Tutorial on EMO
y2
y1
y1
set-oriented
scaling-independent
Dominance-Based Ranking
Eckart Zitzler
ETH Zrich
Types of information:
dominance rank
dominance count
dominance depth
Examples:
MOGA, NPGA
dominance rank
NSGA/NSGA-II
dominance depth
SPEA/SPEA2
MOMH 2002
Tutorial on EMO
Diversity Preservation
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
Nearest neighbor
SPEA2
Histogram
PAES
density estimate
=
sum of f values
where f is a
function of the
distance
density estimate
=
volume of the
sphere defined by
the nearest
neighbor
density estimate
=
number of
solutions in the
same box
f
f
Eckart Zitzler
ETH Zrich
Environmental Selection
new
population
offspring
MOMH 2002
Tutorial on EMO
new
population
archive
new
archive
Selection criteria:
Dominance: only nondominated solutions are kept
Density: less crowded regions are preferred to crowded regions
Time: old archive members are preferred to new solutions
Chance: each solution has the same probability to enter the archive
Overview
Eckart Zitzler
ETH Zrich
The Challenge:
Standard interface for search algorithms
The End:
Conclusions and outlook
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
Step 1:
Step 2:
Step 3:
Step 4:
Step 5:
Step 6:
SPEA2 Algorithm
Generate initial population P0 and empty archive
(external set) A0. Set t = 0.
Calculate fitness values of individuals in Pt and At.
At+1 = nondominated individuals in Pt and At.
If size of At+1 > N then reduce At+1, else if
size of At+1 < N then fill At+1 with dominated
individuals in Pt and At.
If t > T then output the nondominated set of At+1.
Stop.
Fill mating pool by binary tournament selection
with replacement on At+1.
Apply recombination and mutation operators to
the mating pool and set Pt+1 to the resulting
population. Set t = t + 1 and go to Step 2.
[Zitzler, Laumanns, Thiele: 2001]
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
y2
y2
SPEA
MOMH 2002
Tutorial on EMO
SPEA2
0
2
5+2
1
0
4
0
4
4+3
5+7
5+4
2+1+4+3+2
y1
4+3+2
y1
S (strength) =
#dominated solutions
S (strength) =
#dominated solutions
R (raw fitness) = N +
strengths of dominators
R (raw fitness) =
strengths of dominators
Eckart Zitzler
ETH Zrich
Diversity Preservation
Density Estimation
MOMH 2002
Tutorial on EMO
Truncation
Incremental approach:
Remove individual A for which
A <d B for all individuals B
<1
Dk = distance to the k-th
nearest individual
k = popsize + archivesize
B <d A iff:
Dk identical for A and B for
all k
Dk of A greater than Dk of B
for a particular k and
identical for smaller k
Overview
Eckart Zitzler
ETH Zrich
The Challenge:
Standard interface for search algorithms
The End:
Conclusions and outlook
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
Limit Behavior
Basic assumptions:
Every solution can be generated from every other solution by
mutation
The number of iterations t goes to infinity (t )
Studies:
Convergence: [Hanne: 1999][Rudolph, Agapie: 2000]
Diversity: e.g., [Knowles, Corne: 2000][Deb et al.: 2001]
Convergence + diversity:
Unlimited memory resources [Rudolph and Agapie: 2000]
Limited memory resources [Laumanns et al.: 2002]
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
Epsilon Dominance
MOMH 2002
Tutorial on EMO
Definition 1: -Dominance
A -dominates B iff f(A) f(B)
(known since 1987)
-dominated
dominated
Pareto front
-Pareto front
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
Algorithm: (-update)
Accept a child if
n the corresponding box is not
dominated by any box that
contains an individual
y2
AND
y1
1
1
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
Variation: single
point mutation
1 1 0 1 0 0 0
1 1 1 1 1 1 1
1 1 1 1 0 0 0
y1
leading 1s
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
insert
into archive
if not dominated
flip
randomly
chosen bit
remove
dominated
from archive
select
individual
from archive
Variant 1: SEMO
Each individual in the
archive is selected with
the same probability
(uniform selection)
Variant 2: FEMO
Select individual with
minimum number of
children
(fair selection)
Eckart Zitzler
ETH Zrich
Results of Analysis
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
Generational Distance
[Van Veldhuizen: 1999]
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
hypervolume
distance
diversity
spread
cardinality
S
432.34
0.3308
0.3637
0.3622
6
T
420.13
0.4532
0.3463
0.3601
5
Eckart Zitzler
ETH Zrich
T better diversity
value than S
T
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
Overview
Eckart Zitzler
ETH Zrich
The Challenge:
Standard interface for search algorithms
The End:
Conclusions and outlook
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
Algorithm designer
comparison to competing
algorithms mandatory
tests on various benchmark
problems necessary
algorithms and applications
become increasingly complex
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
Algorithms
MOMH 2002
Tutorial on EMO
Applications
SPEA2
knapsack
NSGA-II
TSP
PAES
network
design
text-based
Platform and programming language independent Interface
for Search Algorithms [Bleuler et al.: 2002]
PISA: Implementation
Eckart Zitzler
ETH Zrich
MOMH 2002
Tutorial on EMO
shared
shared
file
filesystem
system
selector
process
application independent:
mating / environmental
selection
individuals are described
by IDs and objective
vectors
text
files
handshake protocol:
state / action
individual IDs
objective vectors
parameters
variator
process
application dependent:
variation operators
stores and manages
individuals
Eckart Zitzler
ETH Zrich
Conclusions
MOMH 2002
Tutorial on EMO
Eckart Zitzler
ETH Zrich
More on EMO
MOMH 2002
Tutorial on EMO
Links:
EMO mailing list:
http://w3.ualg.pt/lists/emo-list/
EMO bibliography:
http://www.lania.mx/~ccoello/EMOO/
PISA website:
http://www.tik.ee.ethz.ch/pisa/
Events:
Conference on Evolutionary Multi-Criterion Optimization (EMO 2003),
April 8-11, 2003, Algarve, Portugal:
http://conferences.ptrede.com/emo03/
Acknowledgments:
Stefan Bleuler, Marco Laumanns, Lothar Thiele