25-29 Juiy.

2004 Budapest, Hungary

Image Fusion using Fuzzy Logic and Applications

Harpreet Singh. Jyoti Raj. Gulsheen Kaur
Department of Electrical and Computer Engineering. Wayne State University, Detroit, MI 48202

Thomas Meitzler
U.S. Army TACOM Survivability Technology Area Warren, MI, 48397-5000

Abstrmt Image fusion deals with integrating data obtained from different sources of information for intelligent systems. Image Fusion provides output as a single image from a set of input images. The fusion should provide a human/machine perceivable result with more useful complete information. A great deal of interest has recently been shown in literature in Lmage Fusion because of its application in automotive, medical and other areas. Some techniques are currently available for image fusion. Some new approaches have been suggested in this work in particular, Fuzzy and Neuro-Fuzzy. Algorithms are also proposed for the above said techniques for image fusion process, The work is also supplemented by algorithms, which help us analyze the output qualitatively on attributes like Entropy, Statistical moments, Uniformity etc.
l i g 1. Fusion Process on a Venn Diagram

1. INTRODUCTION

Sensor fusion is being used extensively in different disciplines. A large number of books and papers are available in the area of sensor fusion. Image fusion is comparatively a new area and is attracting the attention in army research and automotive. Fuzzy Logic Approach has extensively been used for a number of applications. We exploit this approach here for obtaining fused images. Image Fusion has become a topic of great interest to a variety of engineers working in different disciplines. It is being used for medical applications so as to get a better image. It's also being researched in automotive industries to enhance the vision of road so as to see a better image during a rainy or a foggy weather. Image fusion has the followin_gadvantages: It improves the reliability by taking care of the redundant information. 2. It improves the capability as it keeps complementary information. See Fig. 1.
1.

Fuzzy logic was proposed by Zadeh [4] and has application in large number of fields. Some of the applications of Fuzzy Logic listed in [6] are in the areas of robotics, automobiles and target detection [SI. The subject of image fusion is developing at a very fast rate as it has applications in a number of different fields including medical sciences.

Image fusion is a sequel to data fusion. A large number of papers are available in the literature in the area of data fusion. A special issue of proceeding IEEE was devoted to data fusion [I]. This issue simply touches the area of image fusion. A number of approaches are currently being discussed in the literature in image fusion. Pyramid approaches to image fusion are studied in [ 2 ] .Wavelet transform to image fusion is described in 131. An extensive article on image fusion is 121.

it's extremely important to develop new techniques for image fusion so as to meet the needs of industry. Fuzzy Logic and Neuro Fuzzy are some of the approaches, which are finding applications in dif'ferent fields. Fuzzy approaches are used where there is uncertainty and no mathematical relations are easily available. Fuzzy Logic and Neuro Fuzzy logic approaches are finding a lot of applications in different branches of engineering. These approaches are comparatively easy and software tools are available so that these approaches could be implemented. Matlab has developed a Fuzzy Logic Tool Box. Fuzzy logic approach has attracted the attention of several investigators in different disciplines. L. A. Zadeh proposed k z z y logic in 1965. Recently. the approach is being utilized in different disciplines. We propose the use of this approach for pixel level image fusion. rhis approach forms an altemative to a large number of conventional approaches, which are based on a host of empirical relations. Empirical approaches are time consuming and result in a low correlation. Fuzzy logic

0-7803-8353-2/04/$20.002004 IEEE 0

337

FUZZ-IEEE 2004

approach is based on simple rules, which are easy to apply and take less time. This approach is becoming more and more popular as is evidenced by a number of recent papers in this area. Neural Network and Fuzzy Logic approach can be used for sensor fusion. Such a sensor fusion could belong to a class of sensor fusion in which case the features could be input and decision could be output. The help of Neuro-fuzzy of fuzzy systems can achieve sensor fusion. The system can be trained from the input data obtained from the sensors. The basic concept is to associate the given sensory inputs with some decision outputs. After developing the system. another group of input data is used to evaluate the performance of the system. Following algorithm and .M file for pixel level image fusion using Fuzzy Logic illustrate the process of defining membership functions and rules for the image fusion process using FIS (Fuzzy Inference System) editor of Fuzzy Logic toolbox in Matlab.
A. F l i i v Logic fipp~c~achImuge Fusion: to

Algorithm for pixel level image fusion using Fuzzy Logic

~71: Read first image in variable M1 and find its size (rows: z l , columns: SI). 0 Read second image in variable M2 and find its size (rows: z2,columns: s2). 0 Variables MI and M2 are images in matrix form where each pixel value is in the range from 0-255. Use Gray CoI ormap. 0 Compare rows and columns of both input images. If the two images are not of the same size, select the portion, which are of same size. 0 Convert the images in column form which has C= zl*sl entries. e Make a fis (Fuzzy) file, Hhich has two input images. e Decide number and type of membership functions for both the input images by tuning the membership functions. Input images in antecedent are resolved to a degree of membership ranging 0 to 255. 0 Make rules for input images, which resolve the two antecedents to a single number from 0 to 255. 0 For num=l to C in steps of one, apply fuzzification using the rules developed above on the corresponding pixel values of the input images which gives a fuzzy set represented by a membership function and results in output image in column format. 0 Convert the column form to matrix form and display the fused image.
e

function Y=fise-fuzzy(M I .M2) %Y = fuse-fuzzy(M1, M2) image fusion with Fuzzy Logic method % MI - input image #I % M2 - input image #2 YO Y -fused image % check inputs YOSize of the two input images should be same [zl S I ] = size(M 1); fz2 s2] = size(M2); if (zl -= z ) j (SI -= s2) 2 error('lnput images are not of same size'): end; %Co I onnap m = gray(256); f=readfis('fuse.fis'); colormap(m); im=evalfis([M I (:) M2(:)],f): % Converting the column into matrix k=l; for i=l : I :zl for j= 1 :1 :s I img(j,i)=im(k); k=ki 1; end end Y=img; %Computing entropy for input and fused images entropy(M 1 ); entropy(M2); entropy(img) Algorithm and .M file for pixel level image fusion using Neuro-Fuzzy (ANFIS) approach is explained as follows.
C. Xeuro-FUz,.?;Approach to Image Fusion:

Algorithm for pixel level image fusion using ANFIS : Read first image in variable M1 and find its size (rows: zl, columns: sl). Read second image in variable M2 and find its size (rows: 22. columns: s2). Variables M I and M2 are images in matrix form where each pixel value is in the range from 0-255. Use Gray Colormap. Compare rows and columns of both input images. If the two images are not of the same size, select the portion. which are of same size. Convert the images in column form which has C= zl*sl entries.

338

25-29 July, 2004 Budapest, Hungary

Form a training data, which is a matrix with three columns and entries in each column are form 0 to 255 in steps of 1. Form a check data. which is a matrix of Pixels of two

111. WsULrs

A. Resztlis with Medicd image

function Y= fuse-antis(M1 .M2) o/oY = fuse-anfis(M I. M2) image fusion with ?/o Neuro Fuzzy method Y fused image % M1 input image #1 M2 - input image rf2 [ZI SI J = size(M1); [as2J = size(M2); if (zl 22) 1 (s 1 s2) % check inputs error('1nput images are not of same size'); end; m =gray(256): [i 1 row i I col] = size(M 1); tmRata=[O: 1:255:0: 1255;O: 1 :255]; trnData=tmData'; Ohusing images as one column chkData=[Ml (:) M2(:)]; epoch-n=20; n umM Fs=5: niffype = 'gaussmf ; % to start training we need FJS structure fismat-gen fis 1(tmRata,n um MFs,mfType); out-fismat==anfis(trnData,fismat,20); im = evalfis(chkData,out-fismat); k-I: for i-1 :1:il row forj~1:l:ilcol img0.i)-im(k): k-k+ I ; end; end: Y-img; %Computing entropy for input and fused images entropy(M I ): entropy(M2); entropy(img)

Fuzzy logic Entropy Variance

1 183.7

N euro-Fuzzy

Statistical Results 5.3081 Entropy

Variance

4.8901 2272.3

B, Results with Remote Images

j =

Innut lmaees

remoteA.bmp

remoteB.bmp

Fuzzy logic
StiltlStlcdl Results

Neuro-Fuzzy
Entropy Variance
6 975

bntrop) Variance

6.575 2901 5

4032 3

The quantifiers of images help us to describe the texture of the image hence produced after fusion. 339

FUZZ-/E 2004

IV. QUALITATIVE ANALYSIS FUSED ~t IMAGES

It is almost impossible to demarcate some unique universally applicable quantitative performance measures for the fusion process. Texture of an image could be used as a guiding criterion for assessing its quality [SI. Moments of the gray-level histogram of an image or region are the best and simplest approaches for describing texture. Variance. an important texture descriptor, another name for the second moment (&), measures the gay-level contrast used to establish the relative smoothness.
I.

V1. CONCLUSION
Fuzzy and Neuro-Fuzzy algorithms have been implemented to fuse a variety of images. The results of fusion process proposed are given in terms of Entropy and Variance. The fusions have been implemented for medical images and remote sensing images. It is hoped that the techniques can be extended for colored images and for fusion of multiple sensor images.

REFERENCES
Dasarathy Belur. "Sensor Fusion Potential Exploitation - Innovative Architectures and Illustrative Applications." in Proceedings of the I vol. 85. NO. I Januat.;.1997. 121 httn://www.rackiiieer.puresnacc.dc/icin~7.pdf [ 3 ] htia:/ww.i*i.eecs.lehieh.cdiiiSPCRl.,~IF/iii~arre fuusion.htm [Ir] L. Zadeh ''Fozzy Sets." Inform. Contr.,vol. 8,no. 3.pp. 338-353,1965. [ 5 ] T. Meitzler, L. Arafeh, I1. Singh and G. Gerhart .- Fuzzy logic approach for computing the probability of target detection in cluttered cnvironments," Op/. Eng., vol. 35. pp. 3623-3636. Dec 1996. 161 1.ahib Amfeh, Harpreet Singh. Sushi1 Putatunda approach to material processing," in Proceedings o August 1999. [7] Thomas Meitzler. David Bednaa, E. J. Sohn, Kimberly Lane. Darryl Bryk, Gulshecn Kaur. Harpreet Singh. Samuel Ebcnstein, Grmgory Smith, Yelena Rodin, James Rankin "Fumy Logic bascd Image Fusion." Aerosensc 2002 Orlando April 2-5, 2002. [8] Rahel C Gonzalejr? Richard E Woods. Digital Image Processing. 3d Ed. Addison Wesley. [9] Yaonon Wang "Multi sensor image frrsion: Concept, Method and Applications," [I01 Image Fusion for Tactical Applications Roben MeDaniel, D a en Scribncr. William Krebs, Penny Warren, Norman Ockman and Jason McCarley [ I l l F t u q logic hased sensor fusion for mine and concealed weapon detection Thomas Meitzler . DarryI Rwk, E.J. Sohn, Kimberly Lane. Jyoti Raj. lfarpreet Singh. [ 121 linage fusion techniqcs for reniotc sensing applications Sinione. Giovanni and Farina. Alfonso and Morahito. Francesco and Scmico. Sebastiano Bruno and Bruzzone. Lorcnzo (U). Technical Report DIT-02-025, Infomlaiica e Telecomunicazioni, University of Trento.

p 2 ( 5 ) = 0 2 ( z ) = x ( z m)'p(z,) -,
f-l

(1)

[I]

The concept of entropy is well known in communication and was proposed by Shannon. Entropy for images is defined in [9], as "The entropy of an image is a measure of information content. It is the average number of nits needed to quantize the intensities in the image. its definition as:

g=O

where p (8) is the probability of gray level (g), and the range o f g is [O. ...L-I]".
V. APPLICATIONS

Image Fusion has become a topic of great interest to a variety of engineers working in different disciplines. The advantages of image fusion range from medicine or navigation to surveillance, fire control, and missile guidance to improve accuracy. Fuzzy approaches are used where there is uncertainty and no mathematical relations are easily available. The fuzzy logic technique for image hsion is now being widely used in Medical research applications so as to get a better image, as is evident from the results shown in topic 4 of this work. It i s also being researched in navigation and automotive industries to enhance the vision of road so as to see a better image during a rainy or a foggy weather. Multi-spectral sensors are increasingly being employed in military applications. Just as in satellite imagery of the earth, multi-spectral data is required in order to extract the maximum amount of infomation from a scene. The military applications include automated target recognition, battlefield surveillance. intelligent mobility. etc. Images of mine fields generated using light of different wavelengths. andor at different time of the day are fused together to closely estimate the probability of a mine existence at a particular location [IO] t111. Remote sensing applications [12] include the fusion of multi-sensor and multi-temporal images of the same site acquired at different times, by using neural networks.

340