Remarks on dynamic load balancing of integer loads and integral graphs

$
Dragan Stevanovic
University of Nis, Faculty of Sciences and Mathematics, Visegradska 33, 18000 Nis, Serbia and University of Primorska,
Institute Andrej Marusic, Muzejski trg 2, 6000 Koper, Slovenia
Abstract
Cvetkovic et al. [Appl. Math. Comput. 217 (2010), 24682480] presented an algorithm for dynamic load
balancing in discrete load model based on the eigenvectors of the adjacency matrix of the processor network.
In case of integral graphs, whose eigenvalues are integers and whose eigenvector basis may be chosen to
consist of integers only, this algorithm executes in integer arithmetic, providing an argument for the use
of integral graphs as suitable candidates for processor networks. It is observed here that this algorithm,
however, is not capable of balancing all integer load distributions.
After specifying a necessary and sucient condition for properly managing all integer load distributions,
it becomes apparent that the proposed algorithm does not actually depend on the integrality of eigenvectors,
but on employing a set of balancing ows carrying a unit load from a xed vertex to each of the remaining
vertices in the graph. Such sets exist for every connected graph and, thus, yield a simple load balancing
algorithm that can be executed in integer arithmetic for every processor network.
Keywords: Dynamic load balancing; Unit size tokens; Integral graphs.
2000 MSC: 05C50, 68M14
1. Introduction
Balancing the work load among processors is important for an ecient use of parallel computers. The
parallel algorithms often produce work load dynamically, and its amount per processor may change dras-
tically during the execution. Hence, the total work load needs to be occasionally rebalanced among the
neighbouring processors in the network. The common approach for dynamic load balancing is the 2-step
model [1], where in the rst step a balancing ow, specifying the exact amounts of load that each processor
has to send to (or receive from) its neighbouring processors, is calculated, while in the second step, the
load elements are migrated accordingly. We further assume that loads are nonnegative integers, that no
load is generated or consumed during the balancing process, and that the network is static and homoge-
neous, meaning that the topology of the network is xed and that processors have equal computing power,
respectively.
Assuming that the global load vector is known, Hu et al [2] showed that the l
2
-optimal balancing ow
may be computed by using Lagrange multipliers and solving a system of linear equations using the conjugate
gradient algorithm. Although the loads are integers, the conjugate gradient algorithm works in oating point
arithmetic and the nal balancing ow is obtained by rounding the loads to be transferred to the nearest
integer.
With the focus on execution in integer arithmetic, Cvetkovic et al [3] presented recently in this journal a
dynamic load balancing algorithm based on the eigenvectors of the processor network. This algorithm should
$
Supported by the research grant 174033 of the Serbian Ministry of Science and Education, the research programme P1-0285
and the research project J1-4021 of the Slovenian Agency for Research.
Email address: dragance106@yahoo.com (Dragan Stevanovic)
Preprint submitted to Applied Mathematics and Computation October 4, 2012
execute in integer arithmetic, provided that the processor network is an integral graph, whose eigenvalues
are integers and whose eigenvector basis may be chosen to consist of integers only. We show in Section 2,
however, that this algorithm may result in balancing ows that contain noninteger values, obstructing the
focus on integer arithmetic.
In Section 3 we further show that, in order to properly manage all integer load distributions, the approach
of Cvetkovic et al [3] has to employ a set of balancing ows carrying a unit load from a xed processor to
any other processor in the network (instead of employing the eigenvectors). Such set of ows exist for
every connected graph, yielding a fast dynamic load balancing algorithm that may be executed in integer
arithmetic for any network. In terms of the l
2
-norm, it gives suboptimal balancing ows compared to the
algorithm of Hu et al [2], but it may use far less network edges for load migration.
1.1. Notation
Let G = (V, E) be a weakly connected, directed graph with the processors V = 1, . . . , n and m = [E[
edges. Let w
i
Z be the load of the processor i V and let w Z
n
be the load vector. Let =
1
n

n
i=1
w
i
be the average load. A vector w Z
n
satisfying w
i
|, ,| represents the uniform load distribution.
Let N 1, 0, +1
nm
be the vertex-edge incidence matrix of G, where the column corresponding to
the edge e E contains 1 for the tail and +1 for the head of e, while the remaining entries of this column
are 0s. For a vertex i V , let in(i) be the set of edges having i as the head and let out(i) be the set of
edges having i as the tail.
Let x Z
E
be a ow on the edges of G. The direction of the ow is given by the directions of edges and
the signs of the entries of x: x
e
> 0 denotes a ow in the direction of edge e, e E, x
e
< 0 in the opposite
direction. The goal of the load balancing algorithm is, therefore, to nd the balancing ow f Z
E
such
that for each i V :
w
i
= w
i
+

ein(i)
f
e

eout(i)
f
e
,
or, in other words, such that
w w = Nf. (1)
2. Algorithm of Cvetkovic et al
A graph is called integral if all eigenvalues of its adjacency matrix are integers. An integral graph,
besides integer eigenvalues, has also an eigenbasis consisting of integer eigenvectors (see, e.g., [3, Algorithm
4.1] and [4]). Further, a regular graph has an all-one vector j as the eigenvector of its largest eigenvalue.
It was argued in [3, Section 4] that regular integral graphs are suitable candidates for processor networks
of parallel computers, as it should be possible to perform the dynamic load balancing in integer arithmetic
for such graphs using the following algorithm [3, Algorithm 4.2]:
Algorithm 1. Let B = v
0
= j, v
1
, . . . , v
n1
of G be an integral eigenbasis of a regular
integral graph G. For each eigenvector v
i
B other than v
0
= j, dene ad hoc an m-dimensional
balancing ow vector f
i
. Represent the dierence between the load vector and the uniform
load distribution vector as a linear combination of the vectors from B. Form the same linear
combination of balancing ow vectors f
i
to obtain the resulting load balancing ow f.
However, the assumption that this algorithm produces an integer balancing ow for each feasible integer
load vector is violated already for the choice of integral eigenbasis and balancing ow vectors of the Petersen
graph used in [3, Example 4.1]. Integral eigenvectors, other than the all-one vector j, and the corresponding
balancing ows are illustrated in [3, Fig. 10], which we reproduce here as Fig. 1(a). These eigenvectors and
the balancing ows are given in the columns of matrices B and F, respectively, after labeling the vertices
2
1
2
3 4
5
6
7
8 9
10
1
2
3
4
5
6
7
10
8
9
11
12
13 14
15
(a) (b)
Figure 1: (a) Reproduction of [3, Fig. 10]; (b) Labeling and orientation of the Petersen graph.
and the edges as indicated in Fig. 1(b):
B =

1 0 1 1 0 1 1 2 2
0 1 1 0 1 1 0 2 1
1 1 0 1 0 1 1 1 0
1 0 1 0 1 1 1 0 1
0 1 0 1 1 0 1 1 2
1 0 0 0 0 1 1 1 1
0 0 0 0 1 0 0 1 0
0 0 0 1 0 0 1 0 0
0 0 1 0 0 1 0 0 0
0 1 0 0 0 0 0 0 1

, F =

1 1 0 0 0 0 0 0 1
1 0 0 0 1 1 0 1 0
0 0 0 1 1 0 0 0 0
0 0 1 1 0 0 1 0 1
0 1 1 0 0 0 0 1 0
0 0 0 1 0 0 0 0 0
0 1 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0
0 0 0 1 0 1 1 1 1
0 0 1 0 0 0 0 1 0
0 1 0 0 0 0 1 0 0
1 0 0 0 0 1 0 0 0
0 0 0 0 1 0 0 0 1

.
Now, for the integer load vector (1, 0, 0, 0, 1, 0, 0, 0, 0, 0)
T
, which is represented uniquely as
1
3
(v
1
v
2
+v
9
),
the above algorithm produces the load balancing ow
b =
1
3
(b
1
b
2
+ b
9
) =
1
3
(1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1)
T
.
3
Hence, in order to balance just two unit loads, this algorithm produces a balancing ow containing thirds
of unit loads along a total of ten edges of the graph, which is clearly out of the integer arithmetic realm.
3. Analyzing and restating the algorithm of Cvetkovic et al
In this section we will analyze the approach used in the above algorithm in order to derive a necessary and
sucient condition for it to yield integer balancing ow. After restating the algorithm from a new viewpoint,
it will become evident that its work does not depend on the integrality of eigenvectors, but instead on a set
of balancing ows carrying a unit of load from a xed vertex to each of the remaining vertices.
Let B be the n (n 1) matrix of integral eigenvectors of the regular integral graph G (excluding the
all-one vector j) and let F be the m (n 1) matrix of the corresponding balancing ow vectors. The
column sums of both N, containing a single 1 and 1 in each column, and B, containing vectors orthogonal
to the all-one vector, are equal to zero, hence, the last rows of both N and B are linear combinations of
their rst n 1 rows. Denote by N

and B

the matrices obtained by deleting the last row from N and B,

respectively. Note also that the matrix B

is invertible, as it contains n 1 linearly independent column

vectors.
The balancing ow vectors in F are balancing the loads given in the columns of B. Since the uniform
load distribution for each column of B is given by the zero vector, we have
B = NF, and consequently, B

= N

F. (2)
Now, let w be the vector of actual processor loads and w the uniform load distribution vector. Since
w and w have equal sums of components, the vector w w has the sum of components equal to zero. Let
(w w)

denote the vector obtained by deleting the last component from w w. Since both w w and the
columns of B have zero component sums, the linear combination of columns of B giving w w is the same
as the linear combination of vectors from B

giving (w w)

. The coecients of this linear combination

are, hence, given by the vector B
1

(w w)

. The main part of Algorithm 1 may, therefore, be described

simply as:
Algorithm 1. For the load vector w and the uniform load distribution w, the balancing ow
vector f is given by f = FB
1

(w w)

.
The requirement that this algorithm produces integer balancing ow f for each integer load vector means
that FB
1

e is an integer vector for each e Z

n1
. The latter holds if and only if, for every 1 i n 1,
FB
1

e
i
is an integer vector for each standard basis vector e
i
having 1 at position i and zeros elsewhere.
However, FB
1

e
i
is just the column i of FB
1

and we may conclude that Algorithm 1 produces integer

balancing ow for each integer load vector if and only if
FB
1

Z
m(n1)
. (3)
Example 1. For matrices B and F from the previous section holds
FB
1

=
1
3

2 3 3 3 3 1 3 2 2
1 0 2 1 2 1 0 1 0
2 1 1 2 2 1 0 0 1
1 1 1 0 2 0 1 1 1
3 2 3 2 2 2 2 2 0
1 1 0 1 1 0 1 1 1
2 2 2 2 1 2 1 1 2
1 0 1 1 1 1 1 1 0
0 0 0 1 0 1 1 0 2
1 0 0 0 0 2 0 1 1
2 1 0 1 1 1 1 0 2
1 0 1 2 1 2 3 1 2
1 1 2 1 0 2 0 1 1
1 0 0 1 0 1 1 1 2
2 1 2 2 3 2 1 3 1

,
explaining the appearance of the thirds of the load units in the resulting balancing ow.
4
For C = FB
1

from (2) we further obtain

N

C = N

FB
1

= B

B
1

= I. (4)
Respecting the necessary and sucient condition (3) for producing integer balancing ows only, Algorithm 1
can now be restated as:
Algorithm 2. If C Z
m(n1)
satises N

C = I, then for the load vector w and the uniform

load distribution w, the balancing ow vector f is given by f = C(w w)

.
Therefore, the resulting balancing ow does not depend either on the eigenvector matrix B or on the
condition that the eigenvectors of G are integer vectors, but on the product C = FB
1

which has to
satisfy (4). Algorithm 2 can be executed in integer arithmetic for every connected graph G, due to the
following
Claim 1. For every weakly connected, directed graph G there exists C Z
m(n1)
satisfying N

C = I.
Proof. Let v be the vertex of G corresponding to the last row of the vertex-edge incidence matrix N. Since
G is weakly connected, for every i V v there exists a path p
i
from v to i in the graph obtained by
replacing all directed edges of G with undirected edges. Further, for i V v let c
i
be an m-dimensional
(-1,0,+1)-vector given by:
(c
i
)
k
=

0, if edge k does not belong to path p

i
1, if path p
i
traverses edge k in its direction
1, if path p
i
traverses edge k in the opposite direction
The vector c
i
, thus, represents the balancing ow for the load vector consisting of load 1 at processor v,
load 1 at processor i and zero elsewhere, and by (1) we have
N

c
i
= e
i
,
where e
i
is the standard basis vector having 1 at position i and zeros elsewhere. The matrix C is then
obtained by setting the column i to c
i
for each i V v.
Example 2. Construction from the previous claim yields the following matrix C for Petersen graph, as
labelled in Fig. 1(b), after deleting vertex 10:
C =

0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0
1 0 0 0 0 0 0 0 0
0 0 0 0 0 1 0 0 0
0 0 1 0 0 1 0 1 0
0 1 0 0 0 0 1 0 1
0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
1 0 0 1 1 0 0 0 0

For the initial load w = (20, 22, 10, 33, 57, 49, 13, 30, 35, 31)
T
, as used in [3, Example 4.1], the balancing ow
produced by Algorithm 2 is
f
2
= C(30j w)

= (0, 0, 0, 3, 10, 19, 1, 20, 5, 0, 0, 8, 20, 0, 20).

Note that, since C has zero rows corresponding to edges 1, 2, 3, 10, 11 and 14, these edges never participate
in the balancing ows.
5
On the other hand, the balancing ow produced by Algorithm 1 is
f
1
= (12, 4, 12, 2, 14, 0, 4, 12, 5, 11, 8, 0, 4, 11, 15),
while the balancing ow produced by the algorithm of Hu et al [2] is
f
H
= (8, 1, 11, 8, 10, 7, 1, 8, 10, 5, 7, 1, 8, 0, 9).
Note also that, since the algorithm of Hu et al [2] produces the nal integer balancing ow by rounding the
real valued solution found by the conjugate gradient algorithm to the nearest integer, it may happen that
the new loads are not uniformly balanced. In this particular case, the balancing ow f
H
makes the loads
of all processors to be equal to 30, except for the processors 1 and 10, whose new loads become 29 and
31, respectively. Nevertheless, the balancing ow f
H
has the smallest l
2
-norm: [[f
H
[[
2
= 28 compared to
[[f
1
[[
2
= 35.16 and [[f
2
[[
2
= 41.95.
4. Conclusions
Algorithm 2 yields a simple and fast load balancing algorithm working in integer arithmetic that, for
each edge of the processor network, reduces to a single scalar multiplication of a column of C corresponding
to the edge and the global imbalance vector. However, the price for the simplicity and the speed of the
load balancing algorithm is paid by not optimizing the amount of communication necessary to distribute
the loads in the network, as the load balancing always uses only the n1 edges belonging to a shortest path
spanning tree of v.
This disadvantage can be somewhat reduced by employing in Algorithm 2 other matrices C

, satisfying
N

= I, which more uniformly use the edges of the network. The set of such matrices is easily character-
ized: from N

(C

C) = 0 it follows that the columns of C

C are integer solutions of the linear system

N

d = 0, which, in terms of the ows, represent those ows d Z

E
satisfying

ein(i)
d
e
=

eout(i)
d
e
for every vertex i V .
Acknowledgement. The author thanks Robert Elsasser for helpful discussion of load balancing topics.
References
[1] R. Diekmann, A. Frommer, B. Monien, Ecient schemes for nearest neighbor load balancing, Parallel Comput. 25 (1999),
789-812.
[2] Y. Hu, R. Blake, D. Emerson, An optimal migration algorithm for dynamic load balancing, Concurrency: Practice &
Experience, 10 (1998), 467-483.
[3] D. Cvetkovic, T. Davidovic, A. Ilic, S. Simic, Graphs for small multiprocessor interconnection networks, Appl. Math.
Comput. 217 (2010), 24682480.
[4] D. Cvetkovic, Graphs with least eigenvalue -2: the eigenspace of the eigenvalue -2, Rend. Sem. Mat. Messina, Ser. II 25
(9) (2003) 63-86.
6