Water Resour Manage (2007) 21:14291447

DOI 10.1007/s11269-006-9092-5

Multi-Colony Ant Algorithm for Continuous

Multi-Reservoir Operation Optimization Problem
M. R. Jalali & A. Afshar & M. A. Mario

Received: 30 December 2005 / Accepted: 31 August 2006 / Published online: 25 November 2006
# Springer Science + Business Media B.V. 2006

Abstract Ant Colony Optimization (ACO) algorithms are basically developed for discrete
optimization and hence their application to continuous optimization problems require the
transformation of a continuous search space to a discrete one by discretization of the
continuous decision variables. Thus, the allowable continuous range of decision variables is
usually discretized into a discrete set of allowable values and a search is then conducted
over the resulting discrete search space for the optimum solution. Due to the discretization
of the search space on the decision variable, the performance of the ACO algorithms in
continuous problems is poor. In this paper a special version of multi-colony algorithm is
proposed which helps to generate a non-homogeneous and more or less random mesh in
entire search space to minimize the possibility of loosing global optimum domain. The
proposed multi-colony algorithm presents a new scheme which is quite different from those
used in multi criteria and multi objective problems and parallelization schemes. The
proposed algorithm can efficiently handle the combination of discrete and continuous
decision variables. To investigate the performance of the proposed algorithm, the wellknown multimodal, continuous, nonseparable, nonlinear, and illegal (CNNI) Fletcher
Powell function and complex 10-reservoir problem operation optimization have been
considered. It is concluded that the proposed algorithm provides promising and comparable
solutions with known global optimum results.
Key words ant colony . optimization . multi-colony . multi-reservoir
M. R. Jalali (*)
Iran University of Science and Technology (IUST) and Mahab Ghodss Consulting Engrs., Tehran, Iran
e-mail: mrjalali@iust.ac.ir
A. Afshar
Department of Civil Engineering and Center of Excellence for Fundamental Studies in Structural
Mechanics, Iran University of Science and Technology, Tehran, Iran
e-mail: a_afshar@iust.ac.ir
M. A. Mario
Hydrology Program and Department of Civil and Environmental Engineering, University of California,
Davis, CA 95616-8628, USA
e-mail: MAMarino@ucdavis.edu

1430

Water Resour Manage (2007) 21:14291447

The following symbols are used in this paper:

BFi
C
D
G

Gkgb
Ii
L
NC
NR
NT
Ri
Si
Smax
Smin
!,
ij

t0
tij(t)
n
bi


kgb
Pij(k,t)
q
q0

benefit function for cell or element i in a trial solution

string
set of costs associated with the options {cij}
set of decision points {di}
Graph (D,L,C)
value of the objective function for the ant with the best
performance within the past total iterations
inflow to the reservoir at time period i
set of options {lij}
number of release intervals (or classes)
number of reservoirs
number of time periods
release at time period i
reservoir storage volume at time period i
maximum storage allowed
minimum storage allowed
parameters that control the relative importance of the
pheromone trail versus a heuristic value
heuristic value representing the cost of choosing option j at
decision point i
pheromone evaporation coefficient
initial value of pheromone
concentration of pheromone on arc (i,j) at iteration t
benefit coefficients of reservoir n at time period i
ant with the best performance within the past total iterations
probability that ant k selects option lij for decision point i
at iteration t
random variable uniformly distributed over [0, 1]
tunable parameter [0, 1]

1 Introduction
Ant colony optimization algorithms (ACO) have been successfully applied to various
combinatorial optimization problems. As heuristic algorithm, ACOs were first proposed by
Dorigo (1992) and Dorigo et al. (1996) as a multi-agent approach to different combinatorial
optimization problems including the traveling salesman and the quadratic assignment
problem. Later Dorigo and Di Caro (1999) introduced a general ant colony optimization
algorithm, namely ant colony metaheuristic, which enabled the algorithm to be applied to
other engineering problems provided that the problem can be properly formulated.
Recently, Dorigo et al. (2000) reported the successful application of ACO algorithms to a
number of benchmark combinatorial optimization problems. Successful application to the
Quadratic Assignment problem (e.g. Maniezzo and Colorni 1999; Gambardella et al.
1999a), the Shortest Common Supersequence problem (Michels and Middendorf 1999), the
Job Shop Scheduling problem (Colorni et al. 1994), or the Resource-Constraint Project

Water Resour Manage (2007) 21:14291447

1431

Scheduling problem (Merkle et al. 2000) have also been reported. Application of ACO
algorithms to water resources problems, however, is of quite recent origin. Abbaspour et al.
(2001) employed ACO algorithms to estimate hydraulic parameters of unsaturated soils.
Maier et al. (2003) used ACO algorithms to find a near global optimal solution to a water
distribution system, indicating that ACO algorithms may form an attractive alternative to
genetic algorithms for optimal design of water distribution systems. Zecchin et al. (2003)
compared the performance of the original ant system (Dorigo et al. 1996) with that of a
minmax ant system, a modified version of the ant system proposed by Sttzle and Hoos
(1997a, 1997b), for optimization of water distribution networks. Simpson et al. (2001)
discussed the selection of parameters employed in ant algorithms for optimizing pipe
network systems. Jalali et al. (2006) employed ACO algorithms to solve the problem of
optimal reservoir operation. Also, Jalali et al. (2005a) introduced an improved version of
the ACO algorithm in single reservoir operation optimization. They included explorer ants,
and a local search technique in a standard ACO algorithm. Linking a transient flow
simulation model in a pressure system with an ACO algorithm, Abbasi (2005) developed an
optimization model for water supply pipe design considering transient flow.
Optimum operation of multi-reservoir systems has received much attention during the
last three decades. Labadie (2004) presented a state-of-the-art review of the optimal
operation of multi-reservoir systems with mathematical and heuristic optimization
algorithms. He discussed some applications of genetic algorithms, artificial neural
networks, and fuzzy-based approach to the multi-reservoir optimization problem; however,
ACO algorithms were not included in the review.
To partially overcome the dimensionality problem in dynamic programming, evolutionary algorithms have been used. There have been several applications of genetic algorithms
(GAs) to multi-reservoir operation problems (Esat and Hall 1994; Fahmy et al. 1994;
Oliveira and Loucks 1997). Esat and Hall (1994) clearly demonstrated the advantages of
GAs over standard dynamic programming techniques in terms of computational requirements. Recently, Wardlaw and Sharif (1999) applied GAs to four-reservoir system
operation, concluding that algorithm with real value coding performs significantly faster
than the one that employs binary coding. They extended the formulation to a more complex
10-reservoir problem. Being at its early stages of development, Honey Bees Mating
Optimization (HBMO) metaheuristic algorithm was applied to a single reservoir operation
problem with promising results (Bozorg Haddad and Afshar 2004).
A suitable approach for coarse grained parallelization is a multi-colony ant algorithm in
which every processor holds a colony of ants (Middendorf et al. 2002). In a multi-colony ant
system, after each generation, the colonies may exchange information about their solutions.
After information exchange, the new pheromone information may be stored in a pheromone
matrix. Limited work has been reported so far where the colonies exchange information in
order to give the colonies a chance to evolve different pheromone matrices and thus search
in different regions of the search space (Middendorf et al. 2002; Calgari 1999).
ACOs are basically developed for discrete optimization and hence their application to
continuous optimization problems require the transformation of a continuous search space
to a discrete one by discretization of the continuous decision variables. Thus, the allowable
continuous range of decision variables is usually discretized into a discrete set of allowable
values and a search is then conducted over the resulting discrete search space for the
optimum solution (Abbaspour et al. 2001).
Up to now, only a few ant approaches for continuous optimization have been proposed in
the literature. The first method called Continuous ACO (CACO) was proposed by Bilchev
and Parmee in 1995, and also later used by some others (Wodrich and Bilchev 1997 and

1432

Water Resour Manage (2007) 21:14291447

Mathur et al. 2000). Other methods include the API algorithm by Monmarche et al. (2000),
and Continuous Interacting Ant Colony (CIAC), proposed by Dreo and Siarry (2002).
Although both CACO and CIAC claim to draw inspiration from the ACO metaheuristic,
they do not follow it closely. All the algorithms add some additional mechanisms (e.g. direct
communication CIAC and API or nest CACO) that do not exist in regular ACO. They also
disregard some other mechanisms that are otherwise characteristic of ACO (e.g. stigmergy
API or incremental construction of solutions all of them). CACO and CIAC are dedicated
strictly to continuous optimization, while API may also be used for discrete problems.
In this paper, a multi-colony scheme is incorporated into the original ACO algorithms to
address the optimum operation of a multi-reservoir system with continuous search space, as
well as investigating their merits in solving continuous-nonseparable, nonlinear, and illegal
(CNNI) optimization problems. The proposed multi-colony scheme is successfully applied
to a 10-reservoir system with continuous search space and the well-known FletcherPowell
function as a CNNI optimization problem.

2 ACO Algorithms: General Aspects

The ACO mimics the natural behavior of a colony of ants. When ants are traveling, they
deposit a substance called pheromone, forming a pheromone trail as an indirect means of
communication. As more ants choose a path to follow, the pheromone on the path builds
up, making it more attractive for other ants to follow.
In the ACO algorithm, artificial ants are permitted to release pheromone while
developing a solution or after a solution has been fully developed, or both. As stated, the
amount of pheromone deposited is made proportional to the goodness of the solution an
artificial ant develops. Rapid drift of all ants toward the same part of the search space is
avoided by including a stochastic component in the choice decision policy and by means of
numerous mechanisms such as pheromone evaporation, explorer ants, and local search.
For the successful application of ACO algorithms to combinatorial optimization
problems, one must project the problem on a graph. Consider a graph G=(D,L,C), in
which D={di} is the set of decision points at which some decisions are to be made, L={lij}
is the set of options j=1...NC, at each decision point i=1...NT, and C={cij} is the set of
costs associated with option L={lij}. A feasible path on the graph is called a solution (8)k
and the path with minimum cost is called the optimum solution (8*)k.
The transition rule used in the original ant system is defined as follows (Dorigo et al. 1996):
8


! h i
>
>
C

ij
>
ij
>
>
< J
if j 2 allowed k
! h i
P

1
Pij k; t
C

ij
ij
>
>
j1
>
>
>
:
0
otherwise
where Pij(k,t) is the probability that ant k selects option lij for decision point i at iteration t;
Cij(t) is the concentration of pheromone on arc (i,j) at iteration t; ij =1/cij is the heuristic value
representing the cost of choosing option j at decision point i; and ! and are two parameters
that control the relative importance of the pheromone trail and heuristic value. The heuristic
value ij is analogous to providing the ants with sight and is sometimes called visibility. This
value, in static problems, is calculated once at the start of the algorithm and is not changed
during the computation.

Water Resour Manage (2007) 21:14291447

1433

Let q be a random variable uniformly distributed over [0,1], and q0 [0,1] be a tunable
parameter. The next node j that ant k chooses to go is (Dorigo and Gambardella 1997):
8
n
o
< arg max Cil t ! il 
if q  q0
l2allowedk
2
j
:
J
otherwise
where J is a value of a random variable selected according to the probability distribution of
Pij(k,t) [Equation (1)]. Equations (1) and (2) provide a probabilistic decision policy to be
used by the ants to direct their search towards the optimal regions of the search space. The
level of stochasticity in the policy and the strength of the updates in the pheromone trail
determine the balance between the exploration of new points in the state-space and the
exploitation of accumulated knowledge (Dorigo and Gambardella 1997). To simulate
pheromone evaporation, the pheromone evaporation coefficient () is defined which
enables greater exploration of the search space and minimizes the chance of premature
convergence to sub-optimal solutions upon completion of a tour by all ants in the colony.
The global trail updating is done as follows:
Cij t 1 1  :Cij t :Cij t

where Cij(t+1) is the amount of pheromone trail on option j of the ith decision point at
iteration t+1; 01 is the coefficient representing the pheromone evaporation and Cij(t)
is the change in pheromone concentration associated with arc (i,j) at iteration t. The amount
of pheromone Cij(t) associated with arc (i,j) is intended to represent the learned desirability
of choosing option j when at decision point i.
Various methods have been suggested for calculating the pheromone changes. The
method used here was originally suggested by Dorigo and Gambardella (1997) in which
only the ant which produced the globally best (gb) solution from the beginning of the trail
is allowed to contribute to pheromone change:
8
k

gb
<


1=G
if i; j 2 tour done by ant kgb
4
Cij t
:0
otherwise

where Gkgb is value of the objective function for ant kgb
, which is the ant with the best
performance within the past total iterations. Other methods of pheromone change and
update may be found elsewhere (Jalali et al. 2006).
Multi colony approach is a good candidate for parallelization in which several colonies
of ants cooperate to find good solutions. Two different approaches have been studied in the
literature, namely heterogeneous and homogeneous. The heterogeneous approach is to have
colonies of ants with a behavior that differs between the colonies. This approach has been
used for multi criteria optimization problems where the colonies work with different
optimization criteria (see e.g. Gambardella et al. 1999b). In homogeneous approach all the
ants show a similar behavior (see e.g. Michels and Middendorf 1999), and information is
exchanged between the colonies after several iterations. Bolondi and Bondaza (1993)
implemented a very fine-grained parallelization in which every processor holds only a
single ant. Better results have been obtained with a coarser grained variant by Dorigo
(2003, unpublished manuscript). Bullnheimer et al. (1998) proposed a parallelization where
an information exchange between contributing ant colonies was done every k iteration(s)
for some fixed k. Their simulations show how much the running time of the algorithm
decreases with an increasing interval between the information exchanges. However it was

1434

Water Resour Manage (2007) 21:14291447

not discussed how it may influence the solution quality. A parallel ant algorithm for the
Quadratic Assignment problem was implemented by Talbi et al. (1999). A fine-grained
master-worker approach was used, where every worker holds a single ant that produces one
solution. Every worker then sends its solution to the master. The master computes the new
pheromone matrix and sends it to the workers. Sttzle (1998) compared the solution quality
obtained by several independent short runs of an ant algorithm with the solution quality of
one long run whose running time equals the total running times of the short runs. Under
some conditions the short runs proved to give better results. Also, they have the advantage
that they can run in parallel and also it is possible to use different parameter values for the
runs. In Michels and Middendorf (1999) model every processor holds one colony of ants
where after every fixed number of iterations and the locally best solution are exchanged
between the colonies. If after exchange process, a colony receives a new solution better
than the its own best solution, the received solution becomes the new best found solution
for this colony. This solution influences the colony because an elitist strategy is applied
where during trail update some pheromone is always put on the trail that corresponds to the
best found solution. Krger et al. (1998, unpublished manuscript) showed that for the
traveling salesman problem (TSP) it is better to exchange only the best solutions found so
far than to exchange whole pheromone matrices and add the received matrices multiplied
by some small factor to the local pheromone matrix.
Different information exchange strategies between the colonies have been studied by
Middendorf et al. (2000, 2002). These strategies were defined as:
(a)

Exchange of globally best solution: The globally best solution is sent to all colonies
where it becomes the new locally best solution.
(b) Circular exchange of locally best solutions: A virtual neighborhood is established
between the colonies so that they form a directed ring. Every colony sends its locally
best solution to its successor colony in the ring. The variable that stores the best found
solution is updated accordingly.
(c) Circular exchange of migrants: As in (b) the processors form a virtual directed ring. In
an information exchange step every colony compares its mb best ants with the mb best
ants of its successor colony in the ring. The mb best of these 2mb ants are then used to
update the pheromone matrix.
(d) Circular exchange of locally best solutions plus migrants: Combination of strategies
(b) and (c).

3 Proposed Multi-Colony Approach

Most of the previous researches on utilization of multi-colony system have concentrated on
information exchange and updating pheromone matrices in a parallel system. This paper
benefits from the inherent potential in multi-colony ant system to approach a continuous
optimization problem. In fact, utilization of a multi-colony system with heterogeneous
discretization scheme with possibly of information exchange between the colonies help to
provide a non-homogeneous and dynamic discretization scheme in the search space. The
main objective is then, to benefit from multi-colony approach, to develop a scheme to
search for real solution in a continuous search space with a non-homogenous discretized
scheme.
As stated by Jalali et al. (2005b), in small scale problems, discrete-refining (DR)
approach may result in near optimum solution in continuous search space. This was in fact

Water Resour Manage (2007) 21:14291447

1435

Decision

Decision

illustrated with few mathematical problems as well as a real world single reservoir
operation problem. However, in complex real world problems with large continuous search
space, it is very often possible to loose the global optimum domain in the early stages of
DR process forever. In the above mentioned single reservoir problem, due to predefined
target release, the probability of such an outcome was quite low, hence the results were
satisfactory. To overcome this problem, a special version of multi-colony algorithm is proposed
which helps to generate a non-homogeneous and more or less random mesh in entire search
space to minimize the possibility of loosing global optimum domain when DR mechanism is
applied. To illustrate the process of information exchange in the proposed multi-colony
algorithm, Figure 1 is presented for a three-colony system. As is clear, each colony has its
own discretization scheme. When information exchange condition is attained, the best
solution of each colony is determined, and for any decision point, new search space is
developed which is bounded by the minimum and maximum values of the decision variable
from the best solution of the colonies. Therefore, a new search space will form which is
smaller and may contain the global optimal. The new search space is then discretized
considering the different discretization pattern of each colony. After any colony information
exchange process, the whole decision space must be re-discretized. To save the best available
solution, in each decision stage, one of the points in new discretization scheme must coincide
with that of the best available solution. Therefore, while generating a more or less random
and non-homogeneous discretization scheme, it is guaranteed that, in each information
exchange stage, the best solution will surely be transformed to the next stage.
The proposed multi-colony algorithm presents a new scheme which is quite different
from those used in multi criteria problems and parallelization schemes. In most of the

State

State
Colony 1

State

t
lu
So

n
io

Decision

st
Be

Decision

Decision

Colony 1

State

State

Colony 2

Colony 2

Decision

Decision

Information Exchange

State

State
Colony 3

Figure 1 Schematic of information exchange in proposed multi-colony ACO algorithm.

Colony 3

1436

Water Resour Manage (2007) 21:14291447

available multi-colony algorithms, in the stage of information exchange, in addition to

exchange of the best solution between the colonies, pheromone matrices are also updated.
In the proposed scheme, which is the most suitable for solution to continuous domain, due
to the limited stages of information exchange, pheromone matrix will be re-initiated for all
colonies after any information exchange.
The proposed algorithm can efficiently handle the combination of discrete and
continuous decision variables. For continuous decision variables, the DR mechanism is
employed whereas for discrete decision variables, it is disregarded.
To investigate the performance of the proposed algorithm, the well-known multimodal,
continuous, nonseparable, nonlinear, and illegal (CNNI) FletcherPowell function was selected as:
f x

n
X

Ai  Bi 2

i1

Ai

n 
X

aij sin a j bij cos aj

j1

Bi

n 
X

aij sin xj bij cos xj

j1

:  xj  :i 1; 2; :::; n
Thirty-dimensional FletcherPowell function has previously been tested with different
algorithms (Bck 1996). As one may note, the minimum value of the function is zero where
xj =!j for all values of j (Figure 5).
Jalali et al. (2005b) approached this function using ACO with pheromone re-initiation
(PRI), partial path replacement (PPR), discrete refining (DR) mechanism, and very
powerful nonlinear programming (NLP) solver (i.e. Lingo software from Lindo
corporation). The best solution found with ACO and classic NLP solver was 21,550 and
89,993.2, respectively.
In this paper three colonies were employed. Specifications of the colonies are presented
in Table I. Flow diagram of the developed scheme including PRI, PPR, DR, and
information exchange between different colonies is presented in Figure 2. As is clear from
Figure 2, combination of previously developed and implemented improving mechanisms
are also included in the new multi-colony algorithm.
Variable information exchange steps were used. The first exchange was scheduled after
10th PRI, therefore, information exchange steps were reduced by three. High rate of
convergence during the initial periods of computation is the main reason for more frequent
employment of PRI mechanism and variable information exchange steps.
Statistical measures of the results obtained for FletcherPowell function are presented in
Table II. Convergence pattern for the three colonies are presented in Figures 3 and 4. The
best result obtained is 9,943.55 which is 117% better than Jalalis previous result (Jalali et al.
2005b). To give an indication of the desirability of the results, values of !j and xj are plotted
in Figure 5, where xj =!j being the global optimal.

Table I Specifications of colonies for FletcherPowell function

Colony no.
No. of ants
No. of classes

1
75
6

2
100
11

3
100
20

Water Resour Manage (2007) 21:14291447

1437

Pheromone initiation and Heuristic

value computation for each decision
at each decision point for all colonies

Building the string of decisions

by all of the ants of colony

PPR mechanism implementation

(some new ant generation)

Colony

Iteration

Fitness value determination for

all of the ants of colony

Selecting the best solution

(ant) of colony

Updating pheromone of each

decision at each decision point of
colony

PRI condition ?

Yes

Pheromone re-initiation for

each decision at each
decision point of colony

No

DR condition ?

Yes

Discrete refining of search

space for each decision
point of colony

No
No

All colonies
are covered ?
Yes
Information
exchange?

Yes

Selecting best solution, Discrete

refining of search space for each
decision point and Pheromone reinitiation for all colonies

No
No

End condition?
Yes
End

Figure 2 Flowchart of proposed multi-colony ACO algorithm.

1438

Water Resour Manage (2007) 21:14291447

Table II Statistical parameters

of the solutions to Fletcher
Powell function by the proposed
multi-colony ACO algorithm
(averaged over 15 runs)

Parameter

Mean
The best
The worst
SD

Colony
1

30,103.99
9,943.55
57,629.16
13,246.98

28,999.91
9,944.80
56,970.92
13,411.36

28,874.90
9,943.55
56,675.63
13,197.46

4 Application to a 10-Reservoir Operation Problem

Consider the following 10-reservoir system (Figure 6) which was first introduced by
Murray and Yakowitz (1979), and most recently was solved by Wardlaw and Sharif (1999)
employing GA with real coding. This problem is complicated not only in terms of size, but
also because of many time dependent constraints on storage. The problem was formulated
such that it remained solvable by linear programming, and Murray and Yakowitz (1979)
presented a solution using constrained differential dynamic programming. The problem is
beyond the capacity of traditional DP and is difficult with variants such as DDDP, but is
relatively simple to solve by linear programming (LP).
The system comprises reservoirs in series and in parallel, and a reservoir may receive
supplies from one or more upstream reservoirs. Operation of the system is to be optimized
over 12 operating periods to maximize hydropower production. Decision variables for the
problem are reservoir releases in each operating period (decision point). Inflows are defined
for each of the most upstream reservoirs, and initial storage and target storages at the end of
the operating period are specified for each reservoir. In addition, there are minimum operating

500000
450000
400000
Colony 1: 6 Classes

Objective Value

350000

Colony 2: 11 Classes

300000

Colony 3: 20 Classes

250000
200000
150000
100000
50000
0
0

500

1000

1500

2000

2500

3000

Iteration

Figure 3 FletcherPowell function value evaluation by proposed multi-colony ACO algorithm (averaged
over 15 runs).

Water Resour Manage (2007) 21:14291447

1439

Objective Value

500000
450000

Colony 1: 6 Classes

400000

Colony 2: 11 Classes

350000

Colony 3: 20 Classes

300000
Information Exchange Steps
250000
200000
150000
100000
50000
0
0

500

1000

1500

2000

2500

3000

Iteration

Figure 4 FletcherPowell function value evaluation by proposed multi-colony ACO algorithm (best of 15 runs).

storages in each reservoir that must be satisfied, as well as constraints on minimum and
maximum reservoir releases. Details are given by Murray and Yakowitz (1979).
The continuity constraints for each reservoir over each operating period i are:
Si1 Si Ii M:Ri

8i

Xj
3
2
1
0
-4

-3

-2

-1

-1

-2
-3

Xj
-4
Global optimum

Best of proposed ACO

Figure 5 Decision variables of the best result of the FletcherPowell function optimization by proposed
multi-colony ACO.

1440

Water Resour Manage (2007) 21:14291447

2
R3

1
R2

R5
4

R1

R6
R8

R4

R9

R7

10

R10
Figure 6 Schematic of 10-reservoir problem.

where Si is the vector of reservoir storages at time i in NR reservoirs; Ii is the vector of

reservoir inflows in time period i to NR reservoirs; Ri is the vector of reservoir releases in time
period i from NR reservoirs; and M is a NRNR matrix of indices of reservoir connections:
3
2
1 0 0 0 0 0 0 0 0 0
7
6
6 0 1 0 0 0 0 0 0 0 0 7
7
6
6 0 0 1 0 0 0 0 0 0 0 7
7
6
7
6
6 0 1 1 1 0 0 0 0 0 0 7
7
6
6 0 0 0 0 1 0 0 0 0 0 7
7
6
7
6
6 0 0 0 0 0 1 0 0 0 0 7
7
6
6 1 0 0 1 1 1 1 0 0 0 7
7
6
7
6
6 0 0 0 0 0 0 0 1 0 0 7
7
6
6 0 0 0 0 0 0 0 1 1 0 7
5
4
0 0 0 0 0 0 1 0 1 1

Water Resour Manage (2007) 21:14291447

1441

1200
1190

Mean

Function Value

1180
1170
1160
1150
1140
1130
1120
1110
1100
1

No. of Colonies

(a) Average
1200
1190

The Best

Function Value

1180
1170
1160
1150
1140
1130
1120
1110
1100
1

No. of Colonies

(b) The best

0.012

Coefficient of Variation

Function Value

0.010
0.008
0.006
0.004
0.002
0.000
1

3
No. of Colonies

(c) Coefficient of variation

Figure 7 Statistical measures of 10-reservoir operation problem versus number of colonies.

1442

Water Resour Manage (2007) 21:14291447

Table III Statistical parameters

of the solutions to 10-reservoir
problem with proposed multicolony ACO algorithm (averaged
over 10 runs)

Parameter

Colony

Mean
The best
The worst
SD

1,185.18
1,192.09
1,180.88
3.60

1,185.18
1,192.39
1,180.73
3.63

1,185.22
1,192.30
1,181.12
3.60

The objective function to be maximized can be written as:

Max

10 X
12
X

bi :Ri

n1 i1

10
X

h
i
n
g n S13 ; f n

n1

where Ri is the release from reservoir n in time period i, S13 is ending storage of reservoir
n
n, f (n) is the target ending storage of reservoir n. The benefit function bi was tabulated by
(n)
Murray and Yakowitz (1979). The penalty function g for deviation from target ending
storage of reservoir n is expressed as:
h
i
h
i2
n
n
g n S13 ; f n 60 S13  f n

for

S13  f n

n 1:::NR

and
h
i
n
gn S13 ; f n 0

for

S13  f n

n 1:::NR

1200

Objective Value

1100

1000

Colony 1: 4 Classes for each reservoir

Colony 2: 6 Classes for each reservoir
Colony 3: 4 Classes for reservoirs 1,2,3,6,8, and 9
5 Classes for reservoirs 4 and 5
10 Classes for reservoirs 7 and 10

900

800
0

500

1000

1500

2000

2500

3000

Iteration

Figure 8 Objective value evaluation of 10-reservoir operation by proposed multi-colony ACO algorithm
(averaged over 10 runs).

Water Resour Manage (2007) 21:14291447

1443

1200

1100
Objective Value

Information Exchange Steps

1000
Colony 1: 4 Classes for each reservoir
Colony 2: 6 Classes for each reservoir
900
Colony 3: 4 Classes for reservoirs 1,2,3,6,8, and 9
5 Classes for reservoirs 4 and 5
10 Classes for reservoirs 7 and 10
800
0

500

1000

1500

2000

2500

3000

Iteration

Figure 9 Objective value evaluation of 10-reservoir operation by proposed multi-colony ACO algorithm
(the best of 10 runs).

Diverse range of possible storages and releases for different reservoirs is a unique
specification of the problem which imposes serious limitation on any discrete algorithm. As
an example, release for reservoir number 3 is bounded by [0.005, 2.12], whereas that of
reservoir number 10 has a wider range of variation (i.e. 0.01, 18.9). Therefore, assigning the
same number of release classes for all reservoirs will not yield suitable results. In fact,
Wardlaw and Sharif (1999) employed a real coding GA to get as close as 99.8% of the
global optimal (1,194.44 unit).
Jalali et al. (2005b) approached the same 10-reservoir problem using ACO with
pheromone re-initiation (PRI), partial path replacement (PPR), and discrete refining (DR)
mechanism, getting as close as 98.7% of the global optimal.
For a highly complex system, definition of an appropriate heuristic function becomes
more and more difficult. In fact, by defining an inappropriate heuristic function one may
mislead the ants by providing them with wrong sight or vision. This was tested defining:
ij BF i :rj

i 1; :::; NR  NT; j 1; :::; NC

in which BFi is the benefit function for cell or element i in a trial solution string and, for
different values of (Jalali et al. 2005b). It was concluded that =0 might be the best
choice for complex systems such as multi-reservoir problems.
In the proposed multi colony algorithm, the final solution and the convergence pattern
may be influenced by the number of colonies for any number of given ants. To test the
impact of number of colonies, the problem was solved for number of colonies ranging from
one to five for 300 ants. Statistical measures of the results for 10 runs are depicted in
Figure 7. Even though the desirable number of colonies depend on the solution domain, for
the problem in hand three colonies may be considered as the best.
For colony number 1, all releases from reservoirs were discretized in four classes
employing 100 ants. Second colony has a population of 150 ants with six discrete levels.

1444

Water Resour Manage (2007) 21:14291447

20

Release

18
16

LP

14

Proposed Multi-Colony ACO

12
10
8
6
4
2

Res. 1

Res. 2

Res. 4 Res. 5 Res. 6

Res. 3

Res. 8

Res. 7

120

108

96

84

72

60

48

36

24

12

0
Res. 9 Res. 10

Period

(a) Reservoir Releases

35

Reservoir Storage

30

LP

25

Proposed Multi-Colony ACO

20
15
10
5

Res. 1

Res. 2

Res. 3

Res. 4 Res. 5 Res. 6

Res. 7

Res. 8

120

108

96

84

72

60

48

36

24

12

0
Res. 9 Res. 10

Period

(b) Reservoir Storages

Figure 10 Comparison between the best result of proposed multi-colony ACO and result of LP model (10reservoir problem).

Third colony with 150 ants has variable discrete classes for different reservoirs.
Specifically, release from reservoir number 7 and 10 were discretized into 10 classes,
reservoirs 4 and 5 into five classes, and remaining reservoirs were discretized into four
classes. Information exchange takes place when all colonies have at least two pheromone
re-initiation experiences. In the process of information exchange, the best solution between
all colonies is selected and used for developing new discretization scheme for each colony.

Water Resour Manage (2007) 21:14291447

1445

At the time of information exchange, pheromone re-initiation mechanism is implemented,

providing a better initial solution for all colonies to start with.
Results obtained for 10 different runs (using different seed numbers) are presented in
Table III for different colonies. The best solution had an objective function of 1,192.39
which is approximated as 99.8% of the known global solution resulted from LP. The best
solutions so far reported by different researchers for the same problem are 1,190.625
(Murray and Yakowitz 1979 using constrained DDP) and 1,190.25 (Wardlaw and Sharif
1999 using GA real coded). Relatively low standard deviation for 10 runs and three
colonies may be considered as a strength point of the proposed algorithm. Rate of
convergence of the results, averaged over 10 runs, for different colonies are presented in
Figure 8. Figure 9 presents the rate of convergence of the best run for different colony
structures, highlighting the information exchange and discrete refining stages. It is quite
clear that after 2,500 iterations using 150 agents, full convergence has been occurred.
Release from different reservoirs and storage at different periods for the best solution are
presented in Figure 10 (a, b). To have a notion of the global optimal, results of the LP
solution are also indicated on the same figures.

5 Concluding Remarks
Poor performance of the ACO algorithms for continuous optimization problem is due to
discretization of the search space. In small-scale problems, discrete refining (DR) approach
may result in near optimum solution in continuous search space. However, in complex real
world problems with large continuous search space, it is very often possible to loose the
global optimum domain in the early stages of DR process forever.
Generating a non-homogeneous and more or less random mesh in the entire search space
may minimize the possibility of loosing global optimum domain when discrete refining
mechanism is applied. The proposed multi-colony algorithm generates a non-homogeneous,
randomly space discretized scheme which efficiently handles the combination of discrete
and continuous decision variables. For continuous decision variables, the DR mechanism is
employed whereas for discrete decision variables, it is disregarded.
Application of the proposed algorithm to well-known multimodal, continuous, nonseperable, nonlinear, and illegal (CNNI) FletcherPowell function and complex 10-reservoir
problem operation optimization revealed its potential to improve final results in highly
complex continuous optimization problems. For the FletcherPowell function, the best result
obtained is 9,943.55 which is 117% better than Jalalis previous result (Jalali et al. 2005b).
The best objective value of the 10-reservoir problem is approximated as 99.8% of the
known global solution.

References
Abbasi H (2005) Ant colony optimization algorithm for supply pipe design considering transient flow. MSc
thesis, Iran University of Science and Technology, Tehran, Iran
Abbaspour KC, Schulin R, van Genuchten MT (2001). Estimating unsaturated soil hydraulic parameters
using ant colony optimization. Adv Water Resour 24(8):827841
Bck T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, New York
Bilchev G, Parmee IC (1995) The ant colony metaphor for searching continuous design spaces. In: Fogarty
TC (ed) Proceedings of the AISB workshop on evolutionary computation, vol 993 of LNCS. Springer,
Berlin Heidelberg New York, pp 2539

1446

Water Resour Manage (2007) 21:14291447

Bolondi M, Bondaza M (1993) Parallelizzazione di un algoritmo per la risoluzione del problema del comesso
viaggiatore. Masters thesis, Politecnico di Milano, Milan, Italy.
Bozorg Haddad O, Afshar A (2004). MBO (Marriage Bees Optimization), a new heuristic approach in
hydrosystems design and operation. In: Proceedings of the 1st international conference on managing
rivers in the 21st century: issues and challenges. Penang, Malaysia, 2123 Sep 2004, pp 499504
Bullnheimer B, Kotsis G, Strauss C (1998) Parallelization strategies for the ant system. In: De Leone R, et al.
(eds) High performance algorithms and software in nonlinear optimization, applied optimization, vol 24.
Kluwer, Dordrecht, pp 87100
Calgari PR (1999) Parallelization of population-based evolutionary algorithms for combinatorial
optimization problems. PhD thesis, Department DInformatique, Ecole Polytechnique Fdrale De
Lausanne
Colorni A, Dorigo M, Maniezzo V, Trubian M (1994) Ant system for job shop scheduling. Belg J Oper Res
34:3953
Dorigo M (1992) Optimization, learning and natural algorithms. PhD thesis, Politecnico di Milano, Milan,
Italy
Dorigo M, Di Caro G (1999). The ant colony optimization metaheuristic. In: Corne D, Dorigo M, Glover F
(eds) New ideas in optimization. McGraw-Hill, London, pp 1132
Dorigo M, Gambardella LM (1997) Ant colony system: a cooperative learning approach to the traveling
salesman problem. IEEE Trans Evol Comput 1(1):5366
Dorigo M, Maniezzo V, Colorni A (1996) The ant system: optimization by a colony of cooperating ants.
IEEE Trans Syst Man Cybern 26:2942
Dorigo M, Bonabeau E, Theraulaz G (2000) Ant algorithms and stigmergy. Future Gener Comput Syst
16:851871
Dreo J, Siarry P (2002) A new ant colony algorithm using the hierarchical concept aimed at optimization of
multiminima continuous functions. In: Dorigo M, Caro GD, Sampels M (eds) Proceedings of the 3rd
international workshop on ant algorithms (ANTS 2002), vol 2463 of LNCS. Springer, Berlin Heidelberg
New York, pp 216221
Esat V, Hall MJ (1994) Water resources system optimization using genetic algorithms. In: Hydroinformatics
94, Proceedings of the 1st international conference on hydroinformatics. Balkema, Rotterdam, The
Netherlands, pp 225231
Fahmy HS, King JP, Wentzel MW, Seton JA (1994). Economic optimization of river management using
genetic algorithms (paper no 943034). International summer meeting, American Society of Agricultural
Engineers, St Joseph, MI
Gambardella LM, Taillard ED, Dorigo M (1999a) Ant colonies for the quadratic assignment Problem. J Oper
Res Soc 50:167176
Gambardella LM, Taillard ED, Agazzi G (1999b) MACS-VRPT: a multiple ant colony system for vehicle
routing problems with time windows. In: Corne D, Dorigo M, Glover F (eds) New ideas in optimization.
McGraw-Hill, pp 6376
Jalali MR, Afshar A, Mario MA (2005a). Improved ant colony optimization algorithm for reservoir
operation (technical report). Hydroinformatics Center, Civil Engineering Department, Iran University of
Science and Technology, Tehran, Iran
Jalali MR, Afshar A (2005b) Semi-continuous ACO algorithms (technical report). Hydroinformatics Center,
Civil Engineering Department, Iran University of Science and Technology, Tehran, Iran
Jalali MR, Afshar A, Mario MA (2006). Reservoir operation by ant colony optimization algorithms. Iran J
Sci Technol 30(B1):107117
Labadie JW (2004) Optimal operation of multireservoir systems: state of-the-art review. J Water Resour Plan
Manage, ASCE 130(2):93111
Maier HR, Simpson AR, Zecchin AC, Foong WK, Phang KY, Seah HY, Tan CL (2003) Ant colony
optimization for design of water distribution systems. J Water Resour Plan Manage 129(3):200209
Maniezzo V, Colorni A (1999) The ant system applied to the quadratic assignment problem. IEEE Trans
Knowl Data Eng 11:769778
Mathur M, Karale SB, Priye S, Jyaraman VK, Kulkarni BD (2000) Ant colony approach to continuous
function optimization. Ind Eng Chem Res 39:38143822
Merkle D, Middendorf M, Schmeck H (2000) Ant colony optimization for resource-constrained project
scheduling. In: Proceedings of the genetic and evolutionary computation conference (GECCO-2000).
Morgan Kaufmann, San Mateo, CA, pp 893900
Michels R, Middendorf M (1999) An ant system for the shortest common supersequence problem. In: Corne D,
Dorigo M, Glover F (eds) New ideas in optimization. McGraw-Hill, New York, pp 5161
Middendorf M, Reischle F, Schmeck H (2000) Information exchange in multi colony ant algorithms. In:
Rolim J (ed) Parallel and distributed computing, proceedings of the 15 IPDPS 2000 workshops, third

Water Resour Manage (2007) 21:14291447

1447

workshop on biologically inspired solutions to parallel processing problems (BioSP3), 15 May 2000,
Cancun, Mexico, LNCS, vol 1800. Springer, Berlin Heidelberg New York, pp 645652
Middendorf M, Reischle F, Schmeck H (2002) Multi colony ant algorithms. Journal of Heuristics 8:305320
Monmarche N, Venturini G, Slimane M (2000) On how Pachycondyla apicalis ants suggest a new search
algorithm. Future Gener Comput Syst 16:937946
Murray DM, Yakowitz S (1979) Constrained differential dynamic programming and its application to multireservoir control. Water Resour Res 15(5):10171027
Oliveira R, Loucks D (1997) Operating rules for multireservoir systems. Water Resour Res 33(4):839852
Simpson AR, Maier HR, Foong WK, Phang KY, Seah HY, Tan CL (2001) Selection of parameters for ant
colony optimization applied to the optimal design of water distribution systems. In: Proceedings of the
international congress on modeling and simulation, Canberra, Australia, pp 19311936
Sttzle T (1998) Parallelization strategies for ant colony optimization. In: Eiben AE, Bck T, Schoenauer M,
Schwefel H-P (eds) Parallel problem solving from Nature-PPSN V, LNCS, vol 1498. Springer, Berlin
Heidelberg New York, pp 722731
Sttzle T, Hoos HH (1997a) The MAXMIN ant system and local search for the traveling salesman problem.
In: Baeck T, Michalewicz Z, Yao X (eds) Proceedings of IEEE-ICEC-EPS 97, IEEE international
conference on evolutionary computation and evolutionary programming conference. IEEE, pp309314
Sttzle T, Hoos HH (1997b) Improvements on the ant system: introducing MAXMIN ant system. In:
Proceedings of the international conference on artificial neural networks and genetic algorithms.
Springer, Berlin Heidelberg New York, pp 245249
Talbi E-G, Roux O, Fonlupt C, Robillard D (1999) Parallel ant colonies for combinatorial optimization
problems. In: Rolim J et al (eds) Parallel and distributed processing, 11 IPPS/SPDP 99 workshops,
LNCS, vol 1586. Springer, Berlin Heidelberg New York, pp 239247
Wardlaw R, Sharif M (1999) Evaluation of genetic algorithms for optimal reservoir system operation. J Water
Resour Plan Manage, ASCE, 125(1):2533
Wodrich M, Bilchev G (1997) Cooperative distributed search: the ants way. Control Cybern (3):413446
Zecchin AC, Maier HR, Simpson AR, Roberts A, Berrisford MJ, Leonard M (2003) Maxmin ant system
applied to water distribution system optimization. In: Modsim 2003 international congress on
modelling and simulation, Modelling and Simulation Society of Australia and New Zealand Inc,
Townsville, Australia, 2, 795800