Feature selection and classification based on ant

colony algorithm for hyperspectral remote sensing

images

Shuang zhou Jun-ping Zhang Bao-ku Su

School of Electronics and School of Electronics and School of Astronautics, Harbin
Information Technology, Harbin Information Technology, Harbin Institute of Technolog
Institute of Technology Institute of Technology Harbin, China
Harbin, China Harbin, China

Abstract—This paper proposes a method of feature selection and
classifcaition based on ant colony algorithm for hyperspectral
remote sensing image. After all features are randomly projected
on a plane, each ant stochastically selects a feature on the plane
firstly, and then decides which route to be selected in terms of the
criterion function among features. Whereafter the feature
combination is formed. At last, using combination feature, the
classification of AVIRIS image is carried out by maximum
likelihood classifier. In order to verify the effectiveness of this
algorithm, the approach is compared with the classical
suboptimal search technique, using AVIRIS images as a data set.
Experimental results prove the processing that based on ant
colony algorithm is more effective and is fit for the band selection
of hyperspectral image.
Keywords-component; feature selection; classification; ant
colony algorithm; hyperspectral remote sensing images
I. INTRODUCTION
Hyperspectral remote sensing provides very high spectral
resolution image data and the potential for discrimination of
subtle differences in ground covers, but the vast amount of data
bring a lot of difficulty to the subsequent interpretation and
analysis. Previous research has demonstrated that high-
dimensional data spaces are mostly empty, indicating that the
data structure involved exists primarily in a subspace. As a
result, there is a need that can reduce the dimensions of the data
to the right number without losing the original information that
allows for the separation of class [1-3].
Either feature selection or feature extraction can reduce data
dimensions. But feature extraction strategies are often complex.
Mostly, aiming at the classification of hyperspectral imaging,
losing of radiation or reverberation information coming from
objects on the ground and contained by original bands may be
produce. The strategy of feature selection can reserve all kinds
of object information on the ground, but the plenty of
hyperspectral imaging bands and the strong correlation
between bands can not make some of the methods about
feature selection perform. Because of the computation or the
assumption that every feature is independence, the ideal
decreasing dimension can not be acquired, the abundant
information contained by hyperspectral imaging can not be
sufficiently mined. Accordingly, studying a method of efficient
feature selection which feature can represent many
multispectral features is necessary[4,5].
There are several feature searching methods for
hyperspectral image. But we can find the limitation of optimal
method and suboptimal method. So we propose a approach
based on ant colony algorithm, in this approach, firstly all
features are randomly projected on a plane, each ant
stochastically selects a feature on the plan, and then decides
which route to be selected in terms of the criterion function
between features. Two criterion functions are selected. Using
hyperspectral remote sensing images (acquired by the airborne
visible/infrared imaging spectrometer [AVIRIS] sensor) as a
data set, and the maximum likelihood classifier to classify the
selected features, experiments are performed.
The paper is organized as follows. Section 2 summarizes
ant colony algorithm. Section 3 gives a detail of the feature
selection based on ant colony algorithm for hyperspectral
remote sensing image. Section 4 validates the effectiveness of
the proposed method. Finally, section 5 concludes this paper.
II. ANT COLONY ALGORITHM
Ant colony algorithm (ACA) is a cooperative search
technique that mimics the foraging behavior of real life ant
colonies. The ants rapidly establish the shortest route from food
source to their nest. Inspired by the food searching behaviors of
ants, Marco Dorigo et al. proposed ant colony algorithm[6]. At
the beginning, ACA builds solution to the traveling salesman
problem (TSP). In 1991, Deneubourg et al. proposed a model
that explains the phenomenon of ants clustering according to
the behavior of ant colonies in clustering their corpses and
sorting their larvae, then, the model is mainly applied to robotic
implementation[7]. Lumer and Faieta in 1994 developed
Deneubourg et al.’s model[8]. In their model a population of

978-1-4244-4131-0/09/$25.00 ©2009 IEEE

ant-agents moving randomly on a grid was able to carry objects ⎧Q
in order to gather them and exploratory data clustering analysis ⎪ if kth ant used path (i, j) in its tour
Δτ ij = ⎨ Lk (3)
is carried out (LF algorithm). A clustering algorithm for
⎪⎩ 0 otherwise
unsupervised learning was presented by H.Azzag[9]. The
algorithm was inspired from the self-assembling behavior
where Q is a constant, and Lk is the length of the tour
observed in real ants where ants progressively become attached
to an existing support and then successively to other attached constructed by ant k. When ant k is in feature i and so far
ants. The algorithm was achieved to clustering numerical constructed the partial solution S, the transition probability of
databases (either artificial or real). Actually, the research of going to feature j is given by:
ACA has been penetrated from the domain of TSP to several
application fields. The research of this algorithm extends from ⎧ τ ijα (t )η ijβ (t )
⎪⎪ if j∈S
the solution of one dimension static optimization problem to
the solution of multi-dimension dynamic combination ∑ β
p ij = ⎨ τ isα (t )η is (t )
⎪ s∈S
(4)
optimization problem, from discrete domain to continuous ⎪⎩ 0 otherwise
domain.
where S is the set of all available paths; that is, path (i, s ) , s
III. FEATURE SELECTION BASED ON ANT COLONY is a feature not yet visited by the ant k. The parameters α and
ALGORITHM FOR HYPERSPECTRAL REMOTE SENSING IMAGES β control the relative importance of the pheromone versus the
Every spectral band of hyperspectral imaging data which is heuristic information η ij , which is given by:
in the high-dimension space may be looked as a feature. So the
feature selection is band selection in the hyperspectral imaging η ij = J ij (5)
that selected subset is clearly reduced the numbers of
dimension of data and reserve the interesting information as where J ij is the criterion function between features.
much as possible. The process discards the redundancy features
from a set of features in order to reduce the number of For the method of feature selection based on ant colony
dimension, i.e. a set of features X which has m (m < n ) algorithm, in the beginning of each iteration, the feature firstly
features is selected from a set of features which has n features, selected is stochastic. With the increase of iteration, ants intent
with the criterion function J(X) up to maximum. to select the same path to travel. Finally, the feature subset
corresponding with the optimal criterion function is the result.
A. Feature selection based on ant colony algorithm The complexity of time of the search process is
T = O (m ⋅ M ⋅ (n − 1)) . The total complexity of total time of the
Let us consider a classification problem in which a set of n
features is available to characterize each process is T = O(m ⋅ M ⋅ (n − 1) ⋅ N C ) with N C cycles。
pattern X = {x1 , x 2 , ", x n } . Firstly, n features are stochastically
projected on a plane. Then each ant stochastically selects a B. Criterion fuction
feature on the plane. Suppose that τ ij (t ) is pheromone In the concrete application of hyperspectral imaging, the
concentration on the path (i, j ) at time t, n is the size of the criterion function that we adopted are the Bhattacharya (BH)
distance and the Jeffries-Matusita (JM) distance. These
feature set X , m is the total number of the ants in the ant distances are the well known distance measures utilized by the
colony. remote sensing community for feature selection in multi-class
At each iteration of the algorithm, pheromone τ ij (t ) problems.
becomes: a. Bhattacharya (BH) distance
τ ij (t + 1) = (1 − ρ )τ ij (t ) + Δτ ij (t , t + 1) (1) n −1 n
J ( B) = ∑ ∑J ij ( B) (6)
m i =1 j = i +1`

Δτ ij (t , t + 1) = ∑ Δτ ijk (t , t + 1) (2) −1
k =1 1 ⎡ Σi + Σ j ⎤ 1 (Σi + Σ j ) / 2
Jij (B) = (μ j − μi )T ⎢ ⎥ (μ j − μi ) + ln 1/ 2 (7)
where, ρ represents the evaporation degree of pheromone 8 ⎣ 2 ⎦ 2 Σ ⋅ Σ 1/ 2
i j
concentration on path (i,j) after one cycle, and 1 − ρ represents
the factor of residual pheromone, in order to prevent the Where n is the number of classes ( n = 9 , for our image
infinite accumulation of pheromone, usually, the value of ρ is set);
taken in [0,1); Δτ ijk (t , t + 1) is the increase of pheromone J ij (B) is the Bhattacharyya distance between the ith and
concentration laid on path (i,j) by kth ant between time t and jth classes;
t+1, and at the initial time, Δτ ij (0) = 0 . μi is the mean vector of the ith class;
Σ i is the covariance matrix of the ith class.
 b. Jeffries-Matusita (JM) distance Experiments using AVIRIS data was carried out to validate
our feature selection algorithms. Overall, 2572 pixels were
n −1 n
selected to form a training set. Each pixel was characterized by
∑∑
− J ij ( B )
JM = 2 2(1 − e ) (8) the 200 features related to the channels of the sensor. On the
i =1 j = i +1` condition of selecting different number of spectral bands,
Where, n is the number of the classes ( n = 9 , for our image separately using BH distance, JM distance as the criterion
function, simulating experiments on these data are performed.
set); J ij (B) is the Bhattacharyya distance between the ith and
The error matrix and the classification accuracies acquired by
jth classes. the ACA algorithm using JM distance based on 15 bands
We assume that there are Gaussian class distributions for selected are showed in table 1.
the computation of the BH distance and the JM distance. As
BH and JM are distance measures, the larger the obtained TABLE I. ERROR MATRIX ACQUIRED BY THE ACA ALGORITHM USING
distance, the larger the expectation of ant colony, the better the JM DISTANCE BASED ON 15 BANDS SELECTED
solution. Ground Classification
C1 C2 C3 C4 C5 C6
truth accruacy
C. Classification of hyperspectral image C1 1347 81 6 0 0 0 93.93%
C2 109 855 4 0 0 0 88.33%
The features selected from the data set are combined to C3 0 7 736 4 0 0 98.53%
form composite image. In terms of the ground truth distribution C4 0 0 7 1285 0 2 99.30%
and the number of classes, after selecting training sample areas C5 0 0 1 0 488 0 99.80%
in the image, the statistical parameters of ground objects are C6 1 0 0 0 0 211 99.53%
acquired through the trained samples. The maximum likelihood On the condition of selecting different number of spectral
classification is carried out to the combination image. bands, separately using BH distance, JM distance as the
criterion function, we made comparison the effectivenesses of
IV. EXPERIMENT RESULTS proposed method and other suboptimal SFFS techniques(See
Fig.3). It is showed in the Fig.3 that the classification accuracy
To testify the effectiveness of the proposed method for using JM distance is higher than that using BH distance based
feature selection, experiments are conducted on an AVIRIS on the SFFS method, especially when the number of the bands
image of mixed agriculture and forestry in Northwestern is up 10 and down 20. And these experiments show that the
Indian, USA recorded in June 1992. The image set was method proposed by this paper is effective in band selection of
composed of 220 spectral bands acquired in the 0.4—2.5μm. hyperspectral image when the number of the selected bands is
The spacial resolution of every image is 20m. Water absorption down 20.
bands were removed, leaving 200 of original 220 bands. A
scene 145×145 pixels in size was selected for our experiments

Classification accuracy
Classification accuracy

(Fig. 1 shows channel 29 of the sensor). There are sixteen

classes in the area which image showed, include corn ground,
soybean ground, grass, pasturage, wood, hay etc. The ground
truth with six land-cover classes is shown in Fig.2.

Number of spectral bands Number of spectral bands

Figure 3. Comparison result of classification accuracy

Figure 1. Band 29th (wavelength range between about 0.68 and 0.69) of the
hyperspectral image utilized in the experiments

C1 Corn-notill
C2 Soybeans-notill
C3 Woods
C4 Wheat
(a) Classified result using (b) Classified result using
C5 Grass/Tree proposed method(BH) proposed method(JM)
C6 Hay-windrowed

Figure 2. Ground truth map with 6 land-cover classes


(c) Classified result using (d) Classified result using
SFFS(BH) SFFS(JM)
Figure 4. Classified result using selected 15 bands
As can be observed from Fig.3 and Fig.4 comparing the
result of the proposed algorithm with those of SFFS algorithm
on the considered data set, one can notice that the proposed
algorithm allowed greater improvements than the SFFS
algorithm which are well-known and widely used in terms of
the solution quality. Especially, considering the significant
range of selected features (from 10 to 20), the proposed method
provide better solutions than SFFS. And these experiments
show that the proposed method by this paper has the
characteristic of strong search ability and is effective in object
identify of hyperspectral image.
REFERENCES
[1] G. M. Petrie, P. G. Heasler. Optimal Band Selection Strategies for
Hyperpectral Data Sets. IEEE 1998, pp: 1582~158
[2] Sebastiano B.Serpico, Lorenzo Bruzzone, “A New Search Algorithm for
Feature Selection in Hyperspectral Remote Sensing Images”, IEEE
Transactions on Geoscience and Remote Sensing. Vol, 39, No. 7, JULY
2001, pp. 1360-1367
[3] D. Korycinski, M. M. Crawford, J. W. Barnes. Adaptive Feature
Selection for Hyperspectral Data Analysis. Volume 5238 of Proc. of the
SPIE, FEBRUARY 2004, pp: 213~225,
[4] Anil Jain Douglas Zongker. “Feature selection: evaluation, application,
and small sample performance”. IEEE Transactions on Pattern Analysis
and Machine Intelligence, 1997, 19(2), pp:153-158
[5] Manoranjan Dash, Huan Liu. Feature Selection for Classification.
Intelligent Data Analysis, 1997, 1 (3), pp :131-156
[6] Dorigo M, Gambardella L M. “Ant colony system: a cooperative
learning approach to the traveling salesman problem”. IEEE
Transactions on Evolutionary Computation, 1997.1(1).pp. 53-66.
[7] Deneubourg J L, Goss S, Franks N, et al. “The dynamics of collective
sorting: root-like ants and antlike robots”. Proceeding of the 1st
International Conference on Simulation of Adaptive Behavior: From
Animals to Animals, 1991. pp.356-363.
[8] Lumer E, Faieta B. “Diversity and adaptation in populations of
clustering ants”. Proceedings of the 3rd international conference on
simulation of adaptive behavior: from animals to animals, 1994.pp.499-
508.
[9] H. Azzag, N. Monmarch, M. Slimane, G. Venturini, et al, “Ant Tree: a
New Model for Clustering with Artificial Ants”, IEEE Congress on
Evolutionary Computation, Australia, Dec 2003.pp. 2642-2647