e

$EVWUDFW We describe in this paper a new hybrid

approach for mathematical function optimization combining
Particle Swarm Optimization (PSO) and Genetic Algorithms
(GAs) using Fuzzy Logic to integrate the results. The new
evolutionary method combines the advantages of PSO and GA
to give us an improved PSO+GA hybrid method. Fuzzy Logic is
used to combine the results of the PSO and GA in the best way
possible. The new hybrid PSO+GA approach is compared with
the PSO and GA methods with a set of benchmark
mathematical functions. The new hybrid PSO+GA method is
shown to be superior than the individual evolutionary methods.


I. INTRODUCTION

W
E describe in this paper a new evolutionary method
combining PSO and GA, to give us an improved PSOGA
hybrid method. We apply the hybrid method to mathematical
Iunction optimization to validate the new approach. Also in
this paper the application oI a Genetic Algorithm (GA) |1|
and Particle Swarm Optimization (PSO) |2| Ior the
optimization oI mathematical Iunctions is considered. In
this case, we are using the Rastrigin`s Iunction,
Rosenbrock`s Iunction, Ackley`s Iunction, Sphere`s Iunction
and Griewank`s Iunction |4||13| to compare the
optimization results between a GA, PSO and PSOGA.
The main motivation oI this method is to combine the
characteristics oI a GA and PSO. We are using a Iuzzy
system Ior decision making between the methods depending
on the results that we are generating. The Iuzzy system is
used to decide and combine the outputs oI both the GA and
PSO, in this way obtaining the best solution to the
optimization problem. The main goal oI the Iuzzy system is
to evaluate the outputs oI the GA and PSO in each
generation. Three Iuzzy rules are used to achieve this goal.
The criterion Ior stopping the method is the maximum
number oI generations.
The paper is organized as Iollows: in part II a description
about the Genetic Algorithms Ior optimization problems is
given, in part III the Particle Swarm Optimization is
presented, in part IV we describe the proposed method
PSOGA and the Iuzzy system, in part V we describe the

1
Division oI Graduate Studies and Research, Tijuana Institute oI
Technology, Tijuana Mexico, (e-mail: epmelinhaIsamx.org,
ocastillohaIsamx.org )
2
Universidad Autonoma de Baja CaliIornia, Tijuana Institute oI
Technology, 12498 Roll Dr. 1272 San Diego CA 92154, USA (e-mail:
Ievriertectijuana.mx, Ivaldez0125yahoo.com.mx)
3
CITEDI-IPN, Tijuana, Mexico (e-mail: orosscitedi.mx)

mathematical Iunctions that were used Ior this research, in
part VI the simulations results are described, in part VII we
can appreciate a comparison between GA, PSO and
PSOGA, in part VIII the conclusions obtained aIter the
study oI the proposed evolutionary computing methods are
presented.


II. GENETIC ALGORITHM FOR OPTIMIZATION

John Holland, Irom the University oI Michigan initiated his
work on genetic algorithms at the beginning oI the 1960s.
His Iirst achievement was the publication oI Adaptation in
Natural and Artificial Svstem

|7| in 1975.
He had two goals in mind: to improve the understanding
oI natural adaptation process, and to design artiIicial systems
having properties similar to natural systems |8|.
The basic idea is as Iollows: the genetic pool oI a given
population potentially contains the solution, or a better
solution, to a given adaptive problem. This solution is not
"active" because the genetic combination on which it relies
is split between several subjects. Only the association oI
diIIerent genomes can lead to the solution.
Holland`s method is especially eIIective because it not
only considers the role oI mutation, but it also uses genetic
recombination, (crossover) |9|. The crossover oI partial
solutions greatly improves the capability oI the algorithm to
approach, and eventually Iind, the optimal solution.
In Iigure 1 the reproduction cycle oI the Genetic
Algorithm is shown.


Fig. 1. The Reproduction cycle


The essence oI the GA in both theoretical and practical
domains has been well demonstrated |1|. The concept oI
A New Evolutionary Method with a Hybrid Approach Combining
Particle Swarm Optimization and Genetic Algorithms using Fuzzy Logic
Ior Decision Making
Fevrier Valdez
1
, Patricia Melin
2
, Oscar Castillo
2
, Oscar Montiel
3



1333
978-1-4244-1823-7/08/$25.00 c 2008 IEEE


applying a GA to solve engineering problems is Ieasible and
sound. However, despite the distinct advantages oI a GA Ior
solving complicated, constrained and multi-objective
Iunctions where other techniques may have Iailed, the Iull
power oI the GA in application is yet to be exploited |12|
|14|. The Simple Genetic Algorithm can be expressed in
pseudo code with the Iollowing cycle:
1. Generate the initial population oI individuals randomly
P(0).
2. While (number generations maximum numbers
generations)
Do:

Evaluation;
Selection;
Reproduction;
Generation ;
}
3. Show results
4. End

III. PARTICLE SWARM OPTIMIZATION
Particle swarm optimization (PSO) is a population based
stochastic optimization technique developed by Eberhart and
Kennedy in 1995, inspired by the social behavior of bird
flocking or fish schooling [3][17].
PSO shares many similarities with evolutionary
computation techniques such as Genetic Algorithms (GA)
|6|. The system is initialized with a population oI random
solutions and searches Ior optima by updating generations.
However, unlike the GA, the PSO has no evolution operators
such as crossover and mutation. In PSO, the potential
solutions, called particles, Ily through the problem space by
Iollowing the current optimum particles |10|.
Each particle keeps track oI its coordinates in the
problem space, which are associated with the best solution
(Iitness) it has achieved so Iar (The Iitness value is also
stored). This value is called pbest. Another "best" value that
is tracked by the particle swarm optimizer is the best value,
obtained so Iar by any particle in the neighbors oI the
particle. This location is called lbest. When a particle takes
all the population as its topological neighbors, the best value
is a global best and is called gbest |18|.
The particle swarm optimization concept consists oI, at
each time step, changing the velocity oI (accelerating) each
particle toward its pbest and lbest locations (local version oI
PSO). Acceleration is weighted by a random term, with
separate random numbers being generated Ior acceleration
toward pbest and lbest locations.
In the past several years, PSO has been successfully
applied in many research and application areas. It is
demonstrated that PSO gets better results in a faster, cheaper
way compared with other methods [11] [17].
Another reason that PSO is attractive is that there are
Iew parameters to adjust. One version, with slight variations,
works well in a wide variety oI applications. Particle swarm
optimization has been used Ior approaches that can be used
across a wide range oI applications, as well as Ior speciIic
applications Iocused on a speciIic requirement |19|.
The pseudo code of the PSO is as follows
For each particle
Initialize particle
End
Do
For each particle
Calculate fitness value
If the fitness value is better than the best fitness value
(pBest) in history
set current value as the new pBest
End
Choose the particle with the best fitness value of all the
particles as the gBest
For each particle
Calculate particle velocity
Update particle position
End
While maximum iterations or minimum error criteria is not
attained.
IV. PSOGA METHOD
The general idea oI the proposed method, which is called
PSOGA is illustrated in Iigure 2. The method can be
described as Iollows:
1. It receives a mathematical Iunction to be optimized
2. It evaluates the role oI both the GA and PSO.
3. A Iuzzy system is responsible Ior receiving the
values resulting Irom step 2.
4. The Iuzzy system decides in the Iinal step the
optimum value Ior the Iunction given in step 1.
5. Repeat the above steps until the termination
criterion oI the algorithm is met.



Fig. 2 The PSOGA scheme

The basic idea oI the PSOGA scheme is to combine the
advantages oI the individual methods using a Iuzzy system.


A. Fu::v Svstem
As can be seen in the proposed hybrid PSOGA method, it
is the internal Iuzzy system structure, than one that has the
1334 2008 IEEE Congress on Evolutionary Computation (CEC 2008)


primary Iunction oI receiving as inputs the results oI the
outputs oI GA and PSO. The Iuzzy system is responsible Ior
integrating and decides which is the best result being
generated at run time oI the PSOGA. It is also responsible
Ior selecting and sending the problem to the PSO or GA
activating or temporarily stopping depending on the results
being generated. Figure 3 shows the membership Iunctions
oI the Iuzzy system that was implemented in this method.
The Iuzzy system was oI Mamdani type because it is more
common in this type oI Iuzzy control and the
deIuzziIication method was the centroid, in this case, we are
using this type oI deIuzziIication because in other papers we
have achieved good results |16| . The membership Iunctions
were triangular in the inputs and outputs as is shown in
Iigure 3. Also, the membership Iunctions were chosen
triangular based on past experiences in this type oI Iuzzy
control. The Iuzzy system has only 3 rules, because in this
case, we are using one input and one output. For example,
one rule is iI error is low then best value is good (view Iigure
4). Figure 5 shows the Iuzzy system`s rules viewer. Figure 6
shows the surIace corresponding Ior this Iuzzy system. The
surIace shows the behavior oI the Iuzzy rules. We can see in
Iigure 6 that when the error is low the best value is near to
zero.



Fig. 3 Fuzzy system membership Iunctions




Fig. 4 Fuzzy system rules



Fig. 5 Fuzzy system rules viewer



Fig. 6 SurIace oI Iuzzy system.

V. MATHEMATICAL FUNCTIONS
In the Iield oI evolutionary computation, it is common to
compare diIIerent algorithms using a large test set,
especially when the test set involves Iunction optimization.
II we compare 3 searching algorithms with all possible
Iunctions, the perIormance oI any 3 algorithms will be, on
average, the same. As a result, attempting to design a
perIect test set where all the Iunctions are present in order to
determine whether an algorithm is better than any other Ior
every Iunction is impossible. The reason is because, when an
algorithm is evaluated, we must look Ior the kind oI
problems where its perIormance is good, in order to
characterize the type oI problems Ior which the algorithm is
suitable. In this way, we have made a previous study oI the
Iunctions to be optimized Ior constructing a test set with Iive
benchmark Iunctions. This allows us to obtain conclusions
oI the perIormance oI the algorithm depending on the type
oI Iunction. The mathematical Iunctions analyzed in this
paper are the Rastrigin`s Iunction, Rosenbrock`s Iunction,
Ackley`s Iunction, Sphere`s Iunction and Griewank`s
Iunction |5| |15|. All Iunctions were evaluated considering 2
variables.

A. Rastrigins Function
The Rastrigin`s Iunction is given by the Iollowing equation:
)) 2 cos( 10 ( 10 ) (
2
1
i
n
i
x x n x f + =

=
(1)
Where the global minima is: x
*
(0, ., 0), f(x
*
) 0
2008 IEEE Congress on Evolutionary Computation (CEC 2008) 1335


In Iigure 7 the Rastrigin`s Iunction is shown, in which it
can be appreciated that there are several local minima.

Fig. 7 Rastrigin`s Iunction

B. Rosenbrocks Function
The Rosenbrock`s Iunction is given by the equation:
| ) 1 ( ) 1 ( 100 | ) (
2 2 2 2
1
1
+ + =

=
i i i
n
i
x x x x f
(2)
Where the global minima is: x
*
(1, ., 1), f(x
*
) 0.
In Iigure 8 the Rosenbrock`s Iunction is shown, in which
it can be appreciated that there are several local minima.

Fig. 8 Rosenbrock`s Iunction

C. Acklevs Function
The Ackley`s Iunction is given by the equation:
) 2 cos( / 1 20 20 ) (
1
1
/ 1 2
1
5 / 1
x e x n e e x f
n
i
n
i
n
i


= =

+ =
(3)
Where the global minima is: x
*
(0, ., 0), f(x
*
) 0.
In Iigure 9 the Ackley`s Iunction is shown, in which it
can be appreciated that there are several local minima.

Fig. 9 Ackley`s Iunction

D. Spheres Function
The Sphere`s Iunction is given by the Iollowing equation:

=
=
n
i
i
x x f
1
2
) ( (4)
Where the global minima is: x
*
(0, ., 0), f(x
*
) 0.
In Iigure 10 the Sphere`s Iunction is shown, in which it
can be appreciated that there are only one global minima.

Fig. 10 Spheres`s Iunction

E. Griewanks Function
The Griewank`s Iunction is given by the equation:

= =
+ =
n
i
i
n
i
i
i x
x
x f
1 1
2
1 ) / cos(
4000
) (
(5)
Where the global minima is: x
*
(0, ., 0), f(x
*
) 0.
In Iigure 11 the Griewank`s Iunction is shown, in which it
can be appreciated that there are only one global minima.

Fig. 11 Griewank`s Iunction

VI. EXPERIMENTAL RESULTS

Several tests oI the PSO, GA and PSOGA algorithms were
made with implementations in the Matlab programming
language.
All the implementations were developed using a
computer with processor AMD turion X2 oI 64 bits that
works to a Irequency oI clock oI 1800MHz, 2 GB oI RAM
Memory and Windows Vista Ultimate operating system.
The results obtained aIter applying the GA, PSO and
PSOGA to the mathematical Iunctions are shown in tables
I, II, III and IV:
1336 2008 IEEE Congress on Evolutionary Computation (CEC 2008)



The Parameters oI Tables I, II and III are as Iollows:

POP Population size
CROS crossover
MUT mutation
BEST Best Fitness Value
MEAN Mean oI 50 tests
DIMDimensions

A. Experimental Results with the Genetic Algorithm (GA)

From Table I it can be appreciated that aIter executing the
GA 50 times, Ior each oI the tested Iunctions, we can see the
better results and their corresponding parameters that were
able to achieve the global minimum with the method. In
Iigure 12 it can be appreciated the experimental results oI
table I. In Iigure 12 it can be appreciated that the genetic
algorithm was not able to Iind the global minimum Ior the
Ackley`s Iunction because the closest obtained value was oI
2.98.

TABLE I
EXPERIMENTAL RESULTS WITH GA
MATHEMATICAL
FUNCTION
POP CROS MUT BEST MEAN
Rastrigin 100 80 2 7.36E-07 2.15E-03
Rosenbrock 150 50 1 2.33E-07 1.02E-05
Ackley 100 80 2 2.981 2.980
Sphere 20 80 1 3.49-07 1.62E-04
Griewank 80 90 6 1.84E-07 2.552E-05




Fig. 12 Experimental results with GA
.
B. Experimental Results with the Particle Swarm
Optimi:ation (PSO)

From Table II it can be appreciated that aIter executing the
PSO 50 times, Ior each oI the tested Iunctions, we can see
the better results and their corresponding parameters that
were able to achieve the global minimum with the method.
In Iigure 13 it can be appreciated the experimental results oI
table II. In Iigure 13 it can be appreciated that the particle
swarm optimization was not able to Iind the global minimum
Ior Ackley`s Iunction because the closest obtained value was
oI 2.98.

TABLE II
EXPERIMENTAL RESULTS WITH PSO
MATHEMATICAL
FUNCTION
POP DIM BEST MEAN
Rastrigin 20 10 2.48E-05 5.47
Rosenbrock 40 10 2.46E-03 1.97
Ackley 30 1 2.98 2.98
Sphere 20 10 4.88E-11 8.26E-11
Griewank 40 20 9.77E-11 2.56E-02


Fig. 13 Experimental results with PSO

C. Experimental Results with PSOGA

From Table III it can be appreciated that aIter executing the
PSOGA 50 times, Ior each oI the tested Iunctions, we can
see the better results and their corresponding parameters that
were able to achieve the global minimum with the method.
In Iigure 14 it can be appreciated the experimental results Ior
table IV. In Iigure 14 it can be appreciated that the PSOGA
was able to Iind the global minimum Ior all test Iunctions in
this paper because the objective value was reached, and in
all cases was approximately 0.
TABLE III
EXPERIMENTAL RESULTS WITH PSOGA
GA PSO RESULTS
MATH POP CROS MUT DIM POP BEST MEAN
Rastri
100 80 5 10 100 1.45E-06 3.05E-04
Rosen
100 80 2 10 100 1.17E-02 1.17E-02
Ackle 100 50 3 10 100 8.42E-04 4.98E-03
Spher 100 50 3 10 10 5.75E-11 1.05E-10
Griew
160 70 2 40 80 7.88E-11 1.07E-07


Fig. 14 Experimental results with PSOGA


2008 IEEE Congress on Evolutionary Computation (CEC 2008) 1337



VII. COMPARISON BETWEEN GA, PSO AND PSOGA

In Table IV the comparison oI the results obtained between
the GA, PSO and PSOGA methods Ior the optimization oI
the 5 proposed mathematical Iunctions is shown. Table IV
shows the results oI Iigure 15, it can be appreciated that the
proposed PSOGA method was better than GA and PSO,
because with this method all test Iunctions were optimized.
In some cases the GA was better, but in table IV and Iigure
15 it can be seen that the better mean values were obtained
with the PSOGA, only in the Sphere Iunction was better
the PSO than the other two methods. Also in the
Rosenbrock`s Iunction was better the GA than the other two
methods.

TABLE IV
COMPARISON BETWEEN THE PROPOSED METHODS

Mathematical
Functions
GA PSO PSOGA Objective
Value
Rastrigin 2.15E-03 5.47 3.05E-04 0
Rosenbrock 1.02E-05 1.97 1.17E-02 0
Ackley 2.980 2.98 4.98E-03 0
Sphere
1.62E-04 8.26E-11 1.05E-10 0
Griewank
2.552E-05 2.56E-02 1.07E-07 0


Fig. 15 Comparison results between the proposed methods


VIII. CONCLUSIONS

The analysis oI the simulation results oI the 3 evolutionary
methods considered in this paper, in this case the Genetic
Algorithm (GA), the Particle Swarm Optimization (PSO),
and PSOGA lead us to the conclusion that Ior the
optimization oI these 5 mathematical Iunctions, in all cases
one can say that the 3 proposed methods work correctly and
they can be applied Ior this type oI problems. But we can
appreciate that Ior the Ackley`s Iunction the GA and PSO
were not able oI reach the global minimum.
AIter studying the 3 methods oI evolutionary computing
(GA, PSO and PSOGA), we reach the conclusion that Ior
the optimization oI these 5 mathematical Iunctions, GA and
PSO evolved in a similar Iorm, achieving both methods the
optimization oI 4 oI the 5 proposed Iunctions, with values
very similar and near the objectives. Also it is possible to
observe that even iI the GA as the PSO did not achieve the
optimization oI the Ackley`s Iunction, this may have
happened because they were trapped in local minima.
However we can appreciate that the proposed hybrid method
in this paper (PSOGA) was able oI optimize all test
Iunctions. Also, in general PSOGA had the better average
optimization values. The advantage the use this method is
that it incorporates a Iuzzy system to improve the
optimization results.
Figure 15 shows the comparison oI the results obtained
Ior these 5 test Iunctions and it can be appreciated that the
values that were taken Irom the tables above mentioned, the
GA and the PSO obtained very good results and was very
little the diIIerence between oI them. But the PSOGA was
able oI optimize the Ackley`s Iunction while that other two
methods were not able to reach the global minimum Ior this
Iunction. Table IV shows the values corresponding to Iigure
15. The advantage to use PSO is that there are few
parameters used for the implementation. The genetic
algorithm uses more parameters for its implementation. The
PSO+GA is a more complex method but is more reliable
because is a hybrid method that combines the PSO and GA,
also uses a fuzzy system for decision and integration of the
final results, therefore in this research was better that the
other two methods.
Table IV shows the Iinal results oI Iigure 15, it can be
appreciated in some cases, the GA was better than the PSO,
Ior example, Ior the Rastrigin`s Iunction, Rosenbrock`s
Iunction and Griewank`s Iunction. In other cases, the PSO
was better than the GA, Ior example, Ior the Sphere`s
Iunction. But as above is mentioned PSOGA was better
because with this method all Iunctions were optimized with
a smaller error.

ACKNOWLEDGMENT
We would like to express our gratitude to CONACYT,
Universidad Autonoma de Baja CaliIornia and Tijuana
Institute oI Technology Ior the Iacilities and resources
granted Ior the development oI this research.
REFERENCES
|1| Man K.F., Tang K.S., and Kwong S., "Genetic Algorithms: Concepts
and Designs", Springer Verlag. 1999.
|2| Eberhart, R. C. and Kennedy, J. A new optimizer using particle swarm
theory. Proceedings oI the Sixth International Symposium on
Micromachine and Human Science, Nagoya, Japan. pp. 39-43, 1995.J.-
G. Lu, 'Title oI paper with only the Iirst word capitalized, J. Name
Stand. Abbrev., in press.
|3| Kennedy, J. and Eberhart, R. C. Particle swarm optimization.
Proceedings oI IEEE International ConIerence on Neural Networks,
Piscataway, NJ. pp. 1942-1948, 1995.
|4| Valdez, F. and Melin P. Parallel Evolutionary Computing using a
cluster Ior Mathematical Function Optimization, NaIips. San Diego
CA USA, 598-602. June 2007.
|5| Germundsson, R. "Mathematical Version 4." Mathematical J. 7, 497-
524, 2000.
|6| D.B. Fogel, 'An introduction to simulated evolutionary optimization,
IEEE transactions on neural networks, vol 5, n 1, Page(s):3 14, jan
1994.
1338 2008 IEEE Congress on Evolutionary Computation (CEC 2008)


|7| Holland J.H., Adaptation in natural and artiIicial system, Ann Arbor,
The University oI Michigan Press, 1975.
|8| Goldberg D., Genetic Algorithms, Addison Wesley, 1988.
|9| Emmeche C., Garden in the Machine. The Emerging Science oI
ArtiIicial LiIe, Princeton University Press, pp. 114. 1994.
|10|Angeline P. J., Using Selection to Improve Particle Swarm
Optimization , In Proceedings 1998 IEEE World Congress on
Computational Intelligence, Anchorage, Alaska, IEEE, 84-89. 1998.
|11|Angeline P.J., Evolutionary Optimization Versus Particle Swarm
Optimization: Philosophy and PerIormance DiIIerences, Evolutionary
Programming VII, Lecture Notes in Computer Science 1447, Springer,
:601-610. 1998.
|12|Back T., Fogel D.B., and Michalewicz Z., (Eds), Handbook oI
Evolutionary Computation. OxIord University Press, 1997.
|13|Montiel O, Castillo O, Melin P, Rodriguez A and Sepulveda R:
'Human evolutionary model: A new approach to optimization. InI.
Sci. 177(10): 2075-2098, 2007.
|14|Castillo O., Valdez F. and Melin P., 'Hierarchical Genetic Algorithms
Ior topology optimization in Iuzzy control systems'. International
Journal oI General Systems, Volume 36, Issue 5., pag 575-591, October
2007.
|15|Valdez F., Melin P. and Castillo O. 'Evolutionary Computing Ior the
Optimization oI Mathematical Iunctions. Analysis and Design oI
Intelligent Systems Using SoIt Computing Techniques. Advances in
SoIt Computing 41. June 2007.
|16|Castillo O, Huesca G, and Valdez F. 'Proceedings oI the International
ConIerence on ArtiIicial Intelligence, IC-AI '04, Las Vegas,
Nevada,USA, Volume 1, pag. 98-104,June 21-24, 2004.
|17|Veeramachaneni K, Osadciw L and Yan W Improving ClassiIier
Fusion Using Particle Swarm Optimization, IEEE Fusion ConIerence,
Italy, July, 2006.
|18|M and Kennedy J. 'The particle swarm-explosion, stability, and
convergence in a multidimensional complex space. IEEE Transactions
on Evolutionary Computation, 6(1):58-73. 2002.
|19|Kennedy J and Mendes R. 'Population structure and particle swarm
perIormance'.Proceeding oI IEEE conIerence on Evolutionary
Computation, pag. 1671-1676. 2002.

2008 IEEE Congress on Evolutionary Computation (CEC 2008) 1339