Swarm intelligence

Swarm intelligence
Collective system capable of accomplishing difficult tasks in
dynamic and varied environments without any external
guidance or control and with no central coordination.
Achieving a collective performance which could not
normally be achieved by an individual acting alone.
Constituting a natural model
distributed problem solving.

particularly

suited

to

Swarm Algorithms
Inspiration from swarm intelligence has led to some
highly successful optimisation algorithms.
Ant Colony (-based) Optimisation a way to solve
optimisation problems based on the way that ants indirectly
communicate directions to each other.
Particle Swarm Optimisation a different way to solve
optimisation problems, based on the swarming behaviour of
several kinds of organisms.

Optimization
General optimization problem:
given f:X,
find xX such that f(x) is minimum

Given a graph with two specified vertices A and B, find a shortest path
shortest path problem, polynomial
from A to B.
Given a set of cities and pairwise distances, find a shortest tour.
traveling salesperson problem,
Given a sequence of amino acids of a protein, find the structure of the
protein.
protein structure prediction problem,

Ant Colony
Optimization

Real ants
Stigmergy
Autocatalyzation
Ant System
Ant Colony System

Overview
Ant

Colony Optimization (ACO) studies

artificial systems that take inspiration
from the behavior of real ant colonies
and which are used to solve discrete
optimization problems.
-Source: ACO website, http://iridia.ulb.ac.be/~mdorigo/ACO/about.html

Ant Colony Optimization (ACO)

Ant Colony Optimization (ACO)

In the real world, ants (initially) wander randomly, and upon finding food
return to their colony while laying down pheromone trails. If other ants
find such a path, they are likely not to keep traveling at random, but instead
follow the trail laid by earlier ants, returning and reinforcing it if they
eventually find food
Over time, however, the pheromone trail starts to evaporate, thus reducing
its attractive strength. The more time it takes for an ant to travel down the
path and back again, the more time the pheromones have to evaporate.
A short path, by comparison, gets marched over faster, and thus the
pheromone density remains high
Pheromone evaporation has also the advantage of avoiding the
convergence to a locally optimal solution. If there were no evaporation at
all, the paths chosen by the first ants would tend to be excessively
attractive to the following ones. In that case, the exploration of the solution
space would be constrained.

 Thus, when one ant finds a good (short) path from the colony to a food
source, other ants are more likely to follow that path, and such positive
feedback eventually leaves all the ants following a single path.
The idea of the ant colony algorithm is to mimic this behavior with
"simulated ants" walking around the search space representing the
problem to be solved.
Ant colony optimization algorithms have been used to produce nearoptimal solutions to the traveling salesman problem.
They have an advantage over simulated annealing and genetic
algorithm approaches when the graph may change dynamically. The ant
colony algorithm can be run continuously and can adapt to changes in
real time.
This is of interest in network routing and urban transportation systems.

ACO Defined
A heuristic optimization method for shortest path and
other optimization problems which borrows ideas
from biological ants.
Based on the fact that ants are able to find shortest
route between their nest and source of food.

Shortest Route
Shortest route is found using pheromone
trails which ants deposit whenever they
travel, as a form of indirect communication

Ant Colony Optimization

Ant foraging Co-operative search by
pheromone trails

When ants leave their nest to search for a food

source, they randomly rotate around an obstacle

Ant Colony Optimization

Ant foraging Co-operative search by
pheromone trails

initially the pheromone deposits will be the same

for the right and left directions

Ant Colony Optimization

Ant foraging Co-operative search by
pheromone trails

When the ants in the shorter direction find a food source,

they carry the food and start returning back, following their
pheromone trails, and still depositing more pheromone.

Ant Colony Optimization

Ant foraging Co-operative search by
pheromone trails

An ant will most likely choose the shortest path when

returning back to the nest with food as this path will have the
most deposited pheromone

Ant Colony Optimization

Ant foraging Co-operative search by
pheromone trails

For the same reason, new ants that later starts out from
the nest to find food will also choose the shortest path.

Ant Colony Optimization

Ant foraging Co-operative search by
pheromone trails

Over time, this positive feedback (autocatalytic)

process prompts all ants to choose the shorter path

Naturally Observed Ant Behavior

All is well in the world of the

ant.

Naturally Observed Ant Behavior

Oh no! An obstacle has blocked our

path!

Naturally Observed Ant Behavior

Where do we go? Everybody, flip a

coin.

Naturally Observed Ant Behavior

Shorter path reinforced.

Stigmergic?
Stigmergy, a term coined by French biologist
Pierre-Paul Grasse, is interaction through the
environment.
Two individuals interact indirectly when one of
them modifies the environment and the other
responds to the new environment at a later time.
This is stigmergy.

Stigmergy in Ants
Ants are behaviorally unsophisticated, but collectively they
can perform complex tasks.
Ants have highly developed sophisticated sign-based
stigmergy
They communicate using pheromones;
They lay trails of pheromone that can be followed by other ants.

If an ant has a choice of two pheromone trails to follow,

one to the NW, one to the NE, but the NW one is stronger
which one will it follow?

Pheromone Trails
Individual ants lay pheromone trails while travelling from
the nest, to the nest or possibly in both directions.
The pheromone trail gradually evaporates over time.
But pheromone trail strength accumulate with multiple
ants using path.

Nest

Food source

Autocatalyzation
What is autocatalytic behavior?

Pheromone Trails
Initial state:
no ants

t=0

t=1
30 ants

D
d=1

15 ants

d = 0.5

10 ants

20 ants

= 30

= 15

C
= 15 = 30

d = 0.5

d=1

15 ants

30 ants

15 ants

15 ants

10 ants

30 ants

20 ants

30 ants

(a)

(b)

(c)

Autocatalyzation
This is why ACO algorithms are
called autocatalytic positive
feedback algorithms!

Ant Colony
Optimization
The Ant System (AS)

Ant Colony Optimisation Algorithms:

Basic Ideas
Ants are agents that:
Move along between nodes in a graph.
They choose where to go based on pheromone strength (and maybe other
things)
An ants path represents a specific candidate solution.
When an ant has finished a solution, pheromone is laid on its path, according
to quality of solution.
This pheromone trail affects behaviour of other ants by `stigmergy

E.g. A 4-city TSP

Initially, random levels of pheromone are scattered on the edges
A

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

An ant is placed at a random node
A

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

The ant decides where to go from that node,
based on probabilities
A
calculated from:
- pheromone strengths,
- next-hop distances.

Suppose this one chooses BC

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

The ant is now at C, and has a `tour memory = {B, C} so he cannot
visit B or C again.
A
B
Again, he decides next hop
(from those allowed) based
on pheromone strength
and distance;
suppose he chooses
CD

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

The ant is now at D, and has a `tour memory = {B, C, D}
There is only one place he can go now:
A
B

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

So, he has nearly finished his tour, having gone over the links:
BC, CD, and DA.
A
B

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

So, he has nearly finished his tour, having gone over the links:
BC, CD, and DA. AB is added to complete the round trip.
A
B

Now, pheromone on the tour

is increased, in line with the
fitness of that tour.

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

A

Next, pheromone everywhere

is decreased a little, to model
decay of trail strength over
time

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

E.g. A 4-city TSP

We start again, with another ant in a random position.
A

Where will he go?

Next , the actual algorithm
and variants.

Pheromone
Ant

C
AB: 10, AC: 10, AD, 30,

D
BC, 40,

CD 20

The ACO algorithm for the TSP

[a simplified version with all essential details]
We have a TSP, with n cities.
1. We place some ants at each city. Each ant then does this:
It makes a complete tour of the cities, coming back to its starting city, using a
transition rule to decide which links to follow. By this rule, it chooses each
next-city at random, but biased partly by the pheromone levels existing at each
path, and biased partly by heuristic information.

2. When all ants have completed their tours, Global Pheromone

Updating occurs.
The current pheromone levels on all links are reduced (I.e. pheromone levels
decay over time).
Pheromone is lain (belatedly) by each ant as follows: it places pheromone on all
links of its tour, with strength depending on how good the tour was.

Then we go back to 1 and repeat the whole process many times, until we reach a
termination criterion.

 A generalized version of the pseudocode for the ACO algorithm with

reference to the TSP is illustrated below:

ACO algorithm

The basic operational flow in Ant Colony Optimization is as follows:

Step 1: Represent the solution space by a construction graph
Step 2: Initialize ACO parameters
Step 3: Generate random solutions from each ants random walk
Step 4: Update pheromone intensities
Step 5: Goto Step 3, and repeat until convergence or a stopping condition is satisfied.

ACO algorithm
The process starts by generating m random ants (solution).
An ant
represents a solution string, with a
selected value for each variable.
An ant is evaluated according to an objective function.
Accordingly, pheromone concentration associated with
each possible route( variable value) is changed in a way to
reinforce good solutions as follows:

Parameters of ACO Algorithm

ij : Pheromone trail of combination (i,j)

ij : Local heuristic of combination (i,j)

: Transition probability of combination (i,j)

: Relative importance of pheromone trail

: Relative importance of local heuristic

: Determines the relative importance of

q0
exploitation versus exploration
: Trail persistence

46
Pij

ACO algorithm

lij is the length of the tour completed by ant i

ACO algorithm Implementation

above.

ACO Pseudo Code

Pseudo code for ACO is shown below:

Pheromone Concentration
Calculations

Pheromone Concentration Calculation

Main parameters at glance

ACO Characteristics

Natural algorithm since it is based on the behavior of real ants in establishing paths from
their colony to source of food and back.

Parallel and distributed since it concerns a population of agents moving simultaneously,

independently and without a supervisor.

Cooperative since each agent chooses a path on the basis of the information, pheromone trails
laid by the other agents, which have previously selected the same path. This cooperative
behavior is also autocatalytic, i.e., it provides a positive feedback, since the probability of
choosing a path increases with the number of agents that previously chose that path.

Versatile that it can be applied to similar versions of the same problem; for example, there is a
straightforward extension from the traveling salesman problem (TSP) to the asymmetric
traveling salesman problem (ATSP).

Robust that it can be applied with minimal changes to other combinatorial optimization
problems such as quadratic assignment problem (QAP) and the jobshop scheduling problem
(JSP).

Similarities and Differences Between Real Ants and Artificial Ants

Similarities
1.

2.

3.
4.
5.

Colony of cooperating individualsBoth real ant colonies and ant algorithms

are composed of a population, or colony of independent individual agents.
They globally cooperate in order to find a good solution to the task under
consideration
Pheromone trail and stigmergyLike real ants, artificial ants change some
aspects of their environment while walking. Real ants deposit a chemical
substance called pheromone on the visited state. Artificial ants will change
some numerical information of the problem state, locally stored, when that
state is visited.
Local moves and the shortest path searching both finds the shortest path
between the origin and the destination.
Transition policyBoth real ants and artificial ones will build solutions by
applying decision making procedures to move through adjacent states.
Deposited amount of pheromoneAmount of pheromone that an artificial
ant will deposit is mostly a function of the quality of the discovered solution.
In nature, some ants behave in a similar way, the deposited amount of
pheromone is highly dependent on the quality of the discovered food source.

Similarities and Differences Between Real Ants and Artificial Ants

Differences
1.

Artificial ants live in a discrete world. All their moves are jumps from one
discrete state to another adjacent one.

2.

Artificial ants have memory, they could remember states that have been
visited already.

3.

Pheromone deposit methodology is significantly different between real and

artificial ants. Timing in pheromone laying is problem dependent and often
does not have similarities with the real ants pheromone deposit methodology.

4.

To improve overall performance, Ant System algorithms could be enriched

with some additional capabilities that cannot be found in real ant colonies.
Most AS contains some local optimization techniques to improve solutions
developed by ants.

ACO and model-based search (MBS)

algorithm
The ant colony optimization (ACO) is a meta-heuristic approach
(problem-independent techniques, not greedy, even accept a temporary deterioration of
the solution, for example, the simulated-annealing technique, which allows them to
explore more thoroughly the solution space and thus to get a hopefully better solution)

for combinatorial optimization problems inspired by the foraging

behavior of the social insects, especially the ants.
ACO algorithms belong to the class of model-based search (MBS)
algorithm.

Ant Colony Optimization Algorithms

Different ant colony optimization algorithms have been proposed. The
original ant colony optimization algorithm is known as Ant System
and was proposed in the early 90s.
Table gives a list of successful variants of Ant Colony Optimization
Algorithms.

Ant Colony Optimization Algorithms

Figure gives a narrow overview on Ant Colony Optimization
Algorithms.
ACO is a class of algorithms, whose first member, called Ant System,
was initially proposed by Colorni, Dorigo and Maniezzo.

 Many special cases of the ACO metaheuristic have

been proposed.

The three most successful ones are:

1. Ant System,
2. Ant Colony System (ACS), and
3. MAX-MIN Ant System (MMAS)

1. ACO - Ant System

2. ACO - Ant Colony System

First major improvement over Ant System
Differences with Ant System:
1. Decision Rule - Pseudorandom proportional rule
2. Local Pheromone Update
3. Best only offline Pheromone Update

1. Pseudorandom proportional rule

Probability for an ant to move from city i to city j

depends on a random variable q uniformly distributed
over [0, 1], and a parameter q0.

If q q0, then, among the feasible components, the

component that maximizes the product ilil is chosen,
otherwise the same equation as in Ant System is used.

This rule favours exploitation of pheromone information

2. Local pheromone update

The local pheromone update is performed by all ants

after each step.

Each ant applies it only to the last edge traversed:

ij= (1 ) ij + 0

where
(0, 1) is the pheromone decay coefficient
0 is the initial value of the pheromone

3. Best only offline pheromone update

Offline pheromone update equation

ij (1 ) ij + ij best

Where
1/Lbest if best ant k travels on edge i, j
ij best =
0 otherwise

Lbest can be set to the length of the best tour found in the
current iteration or the best solution found since the start
of the algorithm.

3. ACO - MAX-MIN Ant System

Differences with Ant System:
Best only offline Pheromone Update
Min and Max values of the pheromone are explicitly
limited
ij is constrained between min and max (explicitly set by
algorithm designer).
After pheromone update, ij is set to max if ij > max and to
min if ij < min

Applications of Ant Colony Optimization

Traveling Salesman Problem, where a salesman must find the shortest route by
which he can visit a given number of cities, each city exactly once.

Quadratic Assignment Problem, the problem of assigning n facilities to n

locations so that the costs of the assignment are minimized.

Job-Shop Scheduling Problem, where a given set of machines and set of job
operations must be assigned to time intervals in such a way that no two jobs are
processed at the same time on the same machine and the maximum time of
completion of all operations is minimized.

Vehicle Routing Problem, the objective is to find minimum cost vehicle routes
such that:
(a) Every customer is visited exactly once by exactly one vehicle;
(b) For every vehicle the total demand does not exceed the vehicle capacity;
(c) The total tour length of each vehicle does not exceed a given limit;
(d) Every vehicle starts and ends its tour at the same position.

Applications of Ant Colony Optimization

Shortest Common Super sequence Problem, wheregiven a set of strings over an

alphabeta string of minimal length that is a super sequence of each string of the
given set has to be found.

Graph-Coloring Problem, which is the problem of finding a coloring of a graph so

that the number of colors used is minimal.

Sequential Ordering Problem, which consists of finding a minimum weight

Hamiltonian path 2 on a directed graph with weights on the arcs and on the nodes,
subject to precedent constraints among the nodes.

Connection-Oriented Network Routing, where all packets of the same session

follow the same path selected by a preliminary setup phase.

Connectionless Network Routing where data packets of the same session can
follow different paths (Internet-type networks).

Advantages of ACO
Inherent parallelism
Positive Feedback accounts for rapid discovery of
good solutions
Efficient for Traveling Salesman Problem and similar
problems
Can be used in dynamic applications (adapts to
changes such as new distances, etc)

Disadvantages of ACO
Theoretical analysis is difficult
Sequences of random decisions (not independent)
Probability distribution changes by iteration
Research is experimental rather than theoretical
Time to convergence uncertain (but convergence is
gauranteed!)

Ant System (AS) the earliest version of ACO

State Transition Probability

Pij

( ij ) ( ij )

)
(

)
il il
lU

Pheromone Update Rule

NA

ijnew ijold ijk

k 1

Q,
k
ij

Q

d ij
k
ij

Q
or k
L
k
ij

71

ASelite
NA

ijnew ijold ijk ije

k 1

new
ij

ASrank

old
ij

e 1

r 1

r
ij

e
ij

Q
e e
L
e
ij

Q
(e r ) r
L
r
ij

72

Ant-Q & Ant Colony System (ACS)

arg max[( il ) ( il ) ]

lU

q q0

Exploitation

q q0

Exploration

( iv ) ( iv )
Piv
( il ) (il )
lU

Local Updating
(Online Updating)
Global Updating
(Offline Updating)

ijnew ijold (1 ) 0

ijnew ijold (1 ) ije

73

Methodology
Max-Min Ant System (MMAS)

ijnew ijold ije

min ij max

ANTS
Pij

ij (1 ) ij

[
lU

new
ij

old
ij

il

(1 ) il ]
NA

k 1

k
ij

k
L
LB
ijk 0 (1
)
Lavg LB
74