Temporal Analysis of Mammograms Based on

Graph Matching

Fei Ma, Mariusz Bajger, and Murk J. Bottema

School of Computer Science, Engineering and Mathematics,

Flinders University, PO Box 2100
Adelaide SA 5001, Australia
{feim,mbajger,murkb}@infoeng.finders.edu.au

Abstract. A method is proposed for detecting masses in screening

mammograms by analyzing changes between current and previous
mammograms. The method uses graph matching in order to circumvent
the problem of registering images of the same breast taken up to three
years apart. Ninety five temporal pairs of images were separated into
a training set (51 pairs) and a testing set (44 pairs). A small increase
in performance, as measured by the area under the ROC curve, was
found for the testing set when detection rates with graph matching were
compared to detection rates without graph matching.

Keywords: temporal analysis, mammogram registration, graph match-

ing, computer-aided mammography.

1 Background
Many well developed methods for detecting masses in screening mammograms
have been reported in the literature. Typically, detection rates are high but
false positive rates are fairly high too. One way to reduce the number of false
positive reports is to compare the current mammogram with a previous mam-
mogram. Such temporal information is used by radiologists (when available) but
computer programs for exploiting this information have not been entirely suc-
cessful. A key stumbling block is image registration; the problem of identifying
objects in one image with the matching objects in the other image. The image
registration problem is severe since the breast changes naturally over time, the
positioning, dosage and acquisition equipment may vary, and most of all, the
deformation of the soft tissue due to compression of the breast at acquisition
varies unpredictably.
To circumvent this problem, graph matching is prosed as an alternative to
image registration. The steps are (1) segment the current and previous mammo-
grams to produce a list of salient or mass-like regions that includes all reasonable
candidate masses, (2) for each image, assign a graph structure to the segmented
regions, and (3) use graph matching to remove salient objects not relevant to
cancer. The premise is that salient objects not related to cancer will appear as
pairs in the current and previous image while salient objects that appear for the

E.A. Krupinski (Ed.): IWDM 2008, LNCS 5116, pp. 158–165, 2008.

c Springer-Verlag Berlin Heidelberg 2008
 Temporal Analysis of Mammograms Based on Graph Matching 159

first time in the current image or ones that have changed markedly in size or
appearance since the previous image, are likely to be associated with cancer.

2 Methods
2.1 Data
The algorithm described below was developed using 51 training pairs of mammo-
grams, 25 of which were CC views and 26 were MLO views. This set contained
21 malignant masses (confirmed by histopathology). A second set of 44 pairs
(19 CC, 25 MLO, 23 malignant masses) became available later in the study and
was used as a test set. The combined 95 cases were selected randomly from
local screening archives from the same time period under the restriction that
the woman was recalled based on the current mammogram and that a previ-
ous mammogram, less than 3 years old, was available that had not resulted in
recall. Images were digitized using a Vidar Diagnostic Pro Advantage digitizer
(48 μm spatial resolution and 12 bit depth) and boundaries for the true malig-
nant masses were drawn by a radiologist with experience in mammography. For
processing, images were downsampled by a factor of 8 × 8 → 1.

2.2 Segmentation
To achieve reliable correspondence between the components of temporal mam-
mograms and effective analysis for the detection of malignant masses, a mam-
mogram segmentation method must be both robust and effective. Robustness
of a segmentation means that a small change in the image should not result in
a big difference of the segmentation. The same object in consecutive mammo-
grams is usually distorted in rigid and nonrigid ways as described in Section 1. A
non-robust segmentation may segment the same structure in the temporal mam-
mograms into totally different objects, thus decreasing the possibility that the
structure is matched. Effectiveness of the segmentation means that the bound-
aries of the isolated objects are close to their true boundaries. An accurately
segmented boundary of an object facilitates better extraction of object features.
In this study, a version of a hierarchical segmentation method [9], [1], which
has previously been adapted for mammography [5], was used for initial segmenta-
tion. This method, known as the adaptive pyramid method (AP) has advantages
over many other segmentation methods in that both local and global informa-
tion is used. Another benefit is that the AP algorithm is faster than other graph
based methods. Although pyramid based segmentation is known to suffer from
shift variance, in a previous investigation of the robustness of AP segmentation
[4], AP was found to be robust for salient objects.
The AP algorithm results in a tessellation of the breast by components. In
this study, the AP was set so that objects of interest with low intensity con-
trast can be segmented. Thus the images are usually over-segmented. In par-
ticular the large, high contrast objects are typically segmented into several
parts. A merging process was used after the segmentation aiming to merge the
160 F. Ma, M. Bajger, and M.J. Bottema

components belonging to the same large high contrast object. Components C1

and C2 were merged if they satisfied the conditions: area(C1 ) + area(C2 ) <
4400, |Var(C1 )/area(C1 ) − Var(C2 )/area(C2 )| ≤ 0.09, |E(C1 ) − E(C2 )| ≤ 20,
circu(C1 , C2 ) ≤ 0.1 ∗ log(2(area(C1 ) + area(C2 ))). Here V(C), E(C) are the in-
tensity variation and mean intensity of component C and circ(C1 , C2 ) is defined
as the proportion of pixels in C1 ∪ C2 that lie in the intersection of C1 ∪ C2 and
the disk centred at the centroid of C1 ∪ C2 and having the same area as C1 ∪ C2 .
Thresholds used in these conditions were set empirically. Two components were
also merged if one was contained in the other.
Since only components representing information relevant to the presence of
breast cancer are of interest, a filter was used to remove some components from
further consideration. The filter was set by thresholding the area, solidity and
aspect ratio of the component. Here solidity = area/(area of convex hull) and
aspect ratio = (major axis length)/(minor axis length).

2.3 Mass-Like Score

Each component of the segmented image (after filtering) was assigned a mass-like
score to indicate how likely the component is a mass. The mass-like score was
utilized in the graph matching process. Nine features, including solidity, aspect
ratio, area, dradi, pri, radi, circularity, c2 and c3, were selected to calculate the
mass-like score. The solidity, aspect ratio and area are the same as defined in the
previous section. dradi is the standard deviation of the radial distance, which is
the distance of edge pixels to the centroid of the component. pri is defined as
|I − J|/|I + J| with I being the mean intensity of the component and J used for
the mean intensity of the pixels outside of the component but within 2 pixels
from the component. radi is the average radial distance. circularity, c2 and c3
are the same as introduced in [10], [11]. circularity is defined as p2 /area with
p being the perimeter of the component. c2 = (E(C) − E(O))2 /(σ(C) + σ(O)),
where C is the set of pixels of the component and  O is the set of pixels outside
the component but within distance R defined  as area/π. σ(X) is the standard
deviation of the intensity of X. c3 = i |H(C, i) − H(O, i)|, where H(X, i) is
the portion of pixels in component X with intensity i. Finally the mass-like score
was defined as the linear combination of the nine features with the coefficients
optimized using linear discriminant analysis on the training data set.
Based on the mass-like score, the number of components were further reduced.
Only the components having mass-like scores in top 50 are kept.

2.4 Graph Matching

After AP segmentation and filtering, a graph was constructed for each mammo-
gram with nodes corresponding to components of the mammogram. Correspon-
dence between the components of temporal mammograms was then realized by
using graph matching to find a common subgraph between graphs of mammo-
grams. Due to the complexity of the mammogram, exact graph isomorphisms
between graphs seldom exists. The complexity of the mammogram also limits
 Temporal Analysis of Mammograms Based on Graph Matching 161

the success of the graph constructed based on the neighborhood relation of com-
ponents. In this study, a weighted complete graph was used to represent each
mammogram.
Many mammogram registration methods are based on a set of control points
and image interpolation [8], [6]. A method to register mammograms based on
the regions of the mammograms was proposed in [2]. In this study, spacial re-
lations between the components are used to build the correspondence between
the components of temporal mammograms. Accurate expression of the spacial
relations must take into account the shape, size, orientation, etc. of the com-
ponents. Reducing a component into one or few representative points can not
fulfill the goal. For this purpose, fuzzy spacial relation representation introduced
in [7] was implemented. In this fuzzy spacial relation representation, four spacial
relations, ”right of”, ”left of”, ”below”, ”above”, are used to express the spacial
relation. The fuzzy spatial relations were associated to the edges of the graphs
as the weights.
To measure the similarity of matched subgraphs and thus to allow comparison
of different solutions, a match cost function was used. In this match cost function,
similarity is defined based on the fuzzy spacial relations.
For a subgraph Hc = {Vc , Ec }, Vc = {vc1 , vc2 , . . . , vcn } and a subgraph Hp =
{Vp , Ep }, Vp = {vp1 , vp2 , . . . , vpn }, suppose vck ∈ Vc correspond to vpk ∈ Vp , the
similarity between vck and vpk in terms of spacial relation μt is defined as

 n

Ct (vc , vp ) =  (μt (vck , vci ) − μt (vpk , vpi ))2 + (μt (vck , Bc ) − μt (vpk , Bp ))2 .
k k

i=1

Here t is one of the four spacial relations and μt (vm , vn ) is the μt spacial rela-
tion between vertices vm and vn . Bc , Bp are the breast boundaries of the corre-
sponding mammograms. By using the fuzzy spacial relations between the breast
boundary and the components, a global reference is established and explicit
alignment of breast boundaries is not required. The final match cost function is
the accumulation of the similarity measures for individual matches
n
 n

Φ(Hc , Hp ) = Cleft (vci , vpi ) + Cright (vci , vpi ) (1)
i=1 i=1
n n

+ Cabove (vci , vpi ) + Cbelow (vci , vpi ).
i=1 i=1

A version of the backtracking algorithm [12] was developed to perform the

graph matching. This algorithm is based on the work proposed in [3]. During the
graph matching, the condition of spacial compatibility of matched components
is relaxed to allow certain degree nonrigid changes in temporal mammograms.
A longer solution containing false matched components is not better than a
short solution containing more correct matches. Here the length of the solution
refers to the number of the pairs contained in the solution. Thus to find the
162 F. Ma, M. Bajger, and M.J. Bottema

a b c d

e f g h

Fig. 1. An example of matching results. (a) and (b) are temporal mammograms. Pairs
(c)(d), (e)(f) and (g)(h) show examples of matched components.

best solution, not only solutions of same length need to be compared, but also
solutions of different lengths must be compared. To solve this problem, first, the
best solution for each length was found. The best solutions of different length
were then combined together to form one final sequence S = S1 ∪ S2 ∪ · · · ∪ Sk ,
where Si is the best solution of length i.
It is obvious that the final sequence of matches S may contain duplicated
and false matches. The final sequence of matches was cleaned to remove dupli-
cates. To reduce number of false matches, fuzzy spacial relations between the
components and the breast boundaries and relative gray level difference were
calculated. To calculate the relative gray level difference, intensities of one mam-
mogram were transformed to match the histogram of the other mammogram.
An example of matching appears in Figure 1.

3 Results
To evaluate the quality of graph matching, the matching results were examined
by one of the authors (F. Ma) based on visual perception. To do this, the exam-
iner first identified several obvious correspondences between the mammograms.
All matches were then classified into ”good”, ”average”, ”poor” and ”unknown”
based on the identified correspondence. An average of 13.2 matches were formed
over all the mammogram pairs. 63.5% of component matches were identified as
”good”, and 23.6% as ”average”. The percentages of ”poor” and ”unknown”
were 10.9% and 2% respectively.
The process of using the AP algorithm to segment a mammogram, assigning
a mass-like score to each component and using this score to classify components
 Temporal Analysis of Mammograms Based on Graph Matching 163

1 1
a b
0.9 0.9

0.8 0.8

0.7 0.7

true positive rate

true positive rate

0.6 0.6

0.5 0.5

0.4 0.4
AZ without matching: 0.79 A without matching: 0.62
Z
0.3 0.3 A with matching: 0.69
A with matching: 0.8 Z
Z
0.2 0.2

0.1 0.1

0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
false positive rate false positive rate

Fig. 2. ROC curves with and without graph matching. (a) training data, (b) test data.

as malignant masses or not, constitutes a mass detection scheme. This scheme

will be referred to as detection without graph matching. Graph matching is
used to improve the performance of the scheme by removing false positives. To
achieve this, candidate masses detected without graph matching are rejected
if the associated component is matched to a similar component in the previous
mammogram. The resulting detection scheme will be referred to as the detection
with graph matching.
To test the contribution of graph matching, separate ROC curves were con-
structed for detection without graph matching and for detection with graph
matching for both the training set and the testing set (Fig. 2). For the train-
ing set, the Az score (area under the ROC curve) with graph matching was
0.80 and 0.79 without graph matching. For the testing set, Az was 0.69 with
graph matching and 0.62 without. In both training set and testing set, the Az
scores were improved with graph matching. However, improvement on training

a b c d

Fig. 3. False rejection of a true mass. (a) current mammogram. (b) previous mammo-
gram. (c) Same as (a) but with the boundary of a true mass marked according to the
AP segmentation results. (d) Same as (b) but with the boundary marked according to
the AP segmentation results. Since the mass was detected in the previous image, the
true mass was rejected as a false positive in the current screening round.
164 F. Ma, M. Bajger, and M.J. Bottema

a b c d

Fig. 4. A failure case in the testing data set. A malignant mass in the right mammo-
gram (b)(d) was matched to a non-mass component of left mammogram (a)(c). Here
(a) and (b) are the original mammograms.

set was not significant. This is due to two malignant masses in the training set
be matched to two components of corresponding previous mammograms and
thus were identified as non-masses (Fig. 3). A malignant mass in the testing set
was also falsely matched to a component and thus was identified as non-mass
(Fig. 4). In the testing set, the AP method failed in segmenting two malignant
masses. The ROC performance in testing set is thus much lower than the per-
formance in the training set.

4 Discussion
For two cases in the training set, and one in the testing set, the segmentation
step identified a bright region in the previous mammogram that was similar in
appearance and location to the true mass in the current mammogram (Fig. 3).
Since a matching mass was found in the previous image, the true mass was
rejected as a false positive. However, since a mass-like object was detected in
the previous image, it is possible that the mass would have been detected at
the previous round if graph matching had been used to compare the image from
the screening round before the previous round. Such an image was not available
for this study and so this could not be tested. Also the true classification of the
object found in the previous image is not known.
In this study, the performance was measured against the performance of the
without-matching scheme. However, the without-matching scheme was not de-
signed for mass detection on its own. A better test would be to compare results
with state-of-the-art mass detection methods on a common set of images. Un-
fortunately, implementations of such methods were not available. Accordingly,
a good estimate of the contribution of graph matching to the reduction of false
positive detections is not yet known.
 Temporal Analysis of Mammograms Based on Graph Matching 165

Acknowledgments. This work is supported by National Breast Cancer Foun-

dation (Australia).

References
1. Jolion, J.M., Montanvert, A.: The adaptive pyramid: A framework for 2d image
analysis. CVGIP: Image Understanding 55(3), 339–348 (1992)
2. Kok-Wiles, S.L., Brady, M., Hignam, R.: Comparing mammogram pairs for the
detection of lesions. In: Proc. IWDM 1998, pp. 103–110 (1998)
3. Krissinel, E.B., Henrick, K.: Common subgraph isomorphism detection by back-
tracking search. Softw. Pract. Exper. 34, 591–607 (2004)
4. Ma, F., Bajger, M., Bottema, M.J.: Robustness of two methods for segmenting
salient features in screening mammograms. In: Digital Image Computing Tech-
niques and Applications (DICTA 2007), 9th Biennial Conference of the Australian
Pattern Recognition Society, December 2007, pp. 112–117 (2007)
5. Ma, F., Bajger, M., Slavotinek, J.P., Bottema, M.J.: Two graph theory based meth-
ods for identifying the pectoral muscle in mammograms. Pattern Recognition 40,
2592–2602 (2007)
6. Marias, K., Behrenbruch, C., Parbhoo, S., Seifalian, A., Brady, M.: A registra-
tion framework for the comparison of mammogram sequences. IEEE Trans. Med.
Imag. 24(6), 782–790 (2005)
7. Miyajima, K., Ralescu, A.: Spatial organization in 2d segmented images: Represen-
tation and recognition of primitive spatial relations. Fuzzy Sets and Systems 65(2-
3), 225–236 (1994)
8. Sallam, M., Bowyer, K.W.: Registration and difference analysis of corresponding
mammogram images. Medical image analysis 3(2), 103–118 (1999)
9. Tanimoto, S., Pavlidis, T.: A hierarchical data structure for picture processing.
Comput. Graphics Image Process 4(2), 104–119 (1975)
10. Timp, S., Karssemeijer, N.: Interval change analysis to improve computer aided
detection in mammography. Medical Image Analysis 10(1), 82–95 (2006)
11. Timp, S., Varela, C., Karssemeijer, N.: Temporal change analysis for characteri-
zation of mass lesions in mammography. IEEE Trans. Med. Imag. 26(7), 945–953
(2007)
12. Ullmann, J.R.: An algorithm for subgraph isomorphism. Journal of the ACM 1(23),
31–42 (1976)