Mathematical Tools in CS - Problem Set 8

Abu Radi Bader *********

December 29, 2017

1 Problem 1
(a) + (b): We show first that λ = k is an eigenvalue and then we show that for
all i ∈ [n], it holds that |λi | ≤ k. Thus, we can conclude that λ1 = k.
 
1
1
• λ = k is an eigenvalue: we show that x =  .  ∈ Rn is an eigenvector with
 
 .. 
1
n n
Acr ·xc = Acr ·1 =
P P P
eigenvalue λ = k, indeed (Ax)r = 1
c=1 c=1 c ∈ neighbors(r)
 
k
k 
G is k−regular
= k. Hence, Ax =  .  = kx.
 
 .. 
k

• For all i ∈ [n], |λi | ≤ k: Let v 6= 0 be some eigenvector of A, that is, there
is λ ∈ R with Av = λv 1 . Let z ∈ [n] be such that |λvz | is maximal, And
assume W.l.o.g that z = 1. We want to show that |λ| ≤ k: it holds that,
n
Ac1 vc =
P P
λv1 = (λv)1 = (Av)1 = vc
c=1 c ∈ neighbors(1)

Note that since |λv1 | is maximal, we get that |v1 | is maximal in |v|. There-
fore, we get from this and the above equality that

P T he triangle0 s inequality P
|λv1 | = | vc | ≤ |vc |
c ∈ neighbors(1) c ∈ neighbors(1)
G is k−regular and |v1 | is maximal
≤ k|v1 |

All in all, we have |λ||v1 | ≤ k|v1 |, that is |λ| ≤ k.

1A is symmetric over the reals, therefore all of its eigenvalues are reals

1
(c) A is symmetric over R, therefore, its eigenvectors form an orthogonal basis.
Now, assume that G has m connected components V1 , ..., Vm . We will show:
1) There are m orthogonal eigenvectors u(1) , ..., u(m) with eigenvalue k each.
Therefore the multiplicity of k is at least m.
2) Any vector u with Au = ku is in Span{u(1) , ..., u(m) }. Therefore the multi-
plicity of k is at most m
(i)
1. For every 1 ≤ i ≤ m, define the c’th coordinate of u(i) to be uc = 1 if c ∈
(i)
Vi and uc = 0 if c ∈ / Vi . In words, the vector u(i) is an indicator to which
vertices are in the Vi component. Its not hard to see that u(1) , ..., u(m) are
independent, indeed since the components partition the graph we get that
for every i 6= j hvi , vj i = 0. Now, what is left to show is that for every
1 ≤ i ≤ m, it holds that Au(i) = ku(i) . This follows from the fact that the
vertex r is in the component Vi all of r’s neighbors are in Vi , indeed
 P
 1 · 1, if r ∈ Vi
n 
(i) c ∈ neighbors(r)
Acr (i)
P
(Au )r = · (u )c = P
c=1 
 1 · 0, if r ∈
/ Vi
c ∈ neighbors(r)
(
k, if r ∈ Vi
= = (ku(i) )r
0, if r ∈
/ Vi

2. Let u be some non-zero vector with Au = ku, fix i ∈ [m] and let r∗ be a
vertex in Vi with maximal |ur∗ |. Now since Au = ku, we get
P
kur∗ = (ku)r∗ = (Au)r∗ = uc
c ∈ neighbors(r ∗ )

Now, since r∗ has k neighbors and since all of r∗ ’s neighbors are in Vi , we

get that the above equality holds iff each uc in the sum equals ur∗ , this
is because ur∗ has bigger magnitude compared to the k uc s. All in all,
we have that ur∗ = uc for every c ∈ neighbors(r∗ ). Repeating a similar
argument on the neighbors of r∗ and so on... we get that for every r ∈ Vi
it holds that ur is the same. Meaning that, for every component Vi the
values of ur where r ∈ Vi , are all equal. Hence, when restricted to Vi , u is
a scaling of u(i) , thus we can write u as αi u(i) .
P
i

(d)

• For the first direction, assume that that λn = −k, that is there is a non-
zero vector u such that Au = −ku, let r be such that | − ku| is maximal.
We have
n
Acr uc =
P P
−kur = (−ku)r = (Au)r = uc
c=1 c ∈ neighbors(r)

2
 Note that the above equality holds iff uc = −ur for every c ∈ neighbors(r).
Applying a similar argument on the neighbors of r, we will get that if t is
a neighbor of a neighbor of r then ut = −(−ur ) = ur . If we keep applying
the same argument till we reach every vertex in r’s connected component
we will get that every vertex v in this component either has uv = ur or
uv = −ur , and if uv1 6= uv2 then there is no edge between v1 and v2 .
Hence, r’s component is bipartite w.r.t 1) vertices v with uv = ur . and 2)
vertices v with uv = −ur .
• For the other direction, assume that G has bipartite connected component,
denote one side of this component by L and the other side by R, and
assume that L has m vertices, while R has s vertices. Assume W.l.o.g
that L = [m], R = [m + s] \ L. Define the vector u ∈ Rn to be such that
the first m coordinates in u are 1, the coordinates from m + 1 up to m + s
are all equal −1 and the rest of the coordinates are all zero. We claim
that u is an eigenvector with eigenvalue −k, indeed
P
– For every 1 ≤ r ≤ m, we have that (Au)r = uc
c ∈ neighbors(r)
r 0 s neighbors are in R P
= −1 = −k.
c ∈ neighbors(r)
P
– For every m + 1 ≤ r ≤ m + s, we have that (Au)r = uc
c ∈ neighbors(r)
r 0 s neighbors are in L P
= 1 = k.
c ∈ neighbors(r)
P
– For every m + s < r, we have that (Au)r = uc
c ∈ neighbors(r)
r 0 s neighbors are not in L∪R P
= 0 = 0.
c ∈ neighbors(r)

Therefore, Au = −ku.

(e) G is bipartite, that is, V is partitioned to two sets L and R such that G0 s
edges are only between R and L. Assume W.l.o.g that L = [m], R = [m + s] \ L.
We show now that for every eigenvector u 6= 0 with eigenvalue λ, there exists an
eigenvector v 6= 0 with eigenvalue −λ: let u be some eigenvector with eigenvalue
λ. Define v as follows:

• For every 1 ≤ r ≤ m: vr = ur .
• For every m < r: vr = −ur .
In words, v agrees with u on the first m coordinates and agrees with −u on the
rest of the coordinates. Its not hard to see that since u 6= 0, then so is v. We
claim that v is an eigenvector with eigenvalue λ, indeed

3
 P r 0 s neighbors are in R
• For every 1 ≤ r ≤ m, we have that (Av)r = vc =
c ∈ neighbors(r)
P P u0 s eigenvalue is λ
−uc = − uc = −(Au)r =
c ∈ neighbors(r) c ∈ neighbors(r)
−(λu)r .
P r 0 s neighbors are in L
• For every m < r, we have that (Av)r = vc =
c ∈ neighbors(r)
P u0 s eigenvalue is λ
uc = (Au)r = (λu)r .
c ∈ neighbors(r)

Now, since agrees with u on the first m coordinates and agrees with −u on the
other coordinates, we conclude that −λ is an eigenvalue of u. BTW, its not
hard to see that the above describes a linear bijection between the eigenspace
of λ and the eigenspace of −λ.

2 Problem 2
To begin with, we prove the following lemma
Lemma 2.1. Let A be an n×n matrix. It holds that rank(A)+dim(E(0, A)) =
n, where E(0, A) is the eigenspace of the the zero eigenvalues of A.

Proof. A non-zero vector v is an eigenvector with eigenvalue λ iff Av = λv iff

(A − λI)v = 0. That is, the eigenspace of the eigenvalue lambda is Ker(A −
λI). By the nullity theorem on the matrix A, we have rank(A) + ker(A) = n.
Choosing λ = 0, we will get that ker(E(0, A)) = ker(A − 0I) = ker(A), now
substituting this back to rank(A) + ker(A) = n, concludes the result.

Back to the question, the adjacency matrix A of Kn is A = J − I, where J is

the all ones matrix. Since, raank(J) = 1, we get by the above lemma that J
has the 0 eigenvalue with multiplicity n − 1. And its not hard to see that the
vector x ∈ Rn which consists of ones, is an eigenvector of J with eigenvalue n.
Now, since Ax = (J − I)x = Jx − Ix = λx − x = x(λ − 1), we get that the
eigenvalues of A are those of J shifted with -1. That is, A has an eigenvalue of
n − 1 with multiplicity 1, and has the eigenvalue −1 with multiplicity n − 1.

3 Problem 3
To begin with, define for each i ∈ [n], #3 (i) to be the number of different paths
from i to itself. We have seen in recitation that #3 (i) = (A3 )i,i . Now,
of length 3P
n
note that i=1 #3 (i) is 6 times the number of triangles, because each triangle is
counter
Pn twice for each
Pof its nodes. That is, thu number of triangles in the
Pngraph
n
is 61 i=1 #3 (i) = 16 i=1 (A3 )i,i . That is, if we show that trace(A3 ) = i=1 λ3i ,
then we are done. We show this as follows: we know that trace(A3 ) equals to
the sum of A3 eigenvalues (with multiplicity), hence, it is sufficient to show

4
that λ31 , λ32 , ..., λ3n are the eigenvalues of A3 . This is easy! Because, if v is an
eigenvector of A with eigenvalue λ, then v is an eigenvector of A3 with eigenvalue
λ3 , indeed
A3 v = A2 Av = A2 λv = λAAv = λ2 Av = λ3 v

4 Problem 4
(a) To begin with, we use the fact the unit circle is compact. Also, note that
f is continuous (f can be thought of as a polynomial of n variables- x1 , ..., xn ).
Now, since f is a continuous function from a compact set to R, we get that
image(f ) is compact! why?
• image(f ) is closed: let (yn ) ⊆ image(f ) be a sequence that converges
to some y. Hence, by definition there is some sequence (xn ) ⊆ S with
f (xn ) = yn for every n. Now, since S is compact, there is a subsequence
xnk that converges to some x ∈ S. Now, by continuity of f , we have

y = limk→∞ ynk = limk→∞ f (xnk ) = f (x), that is y ∈ image(f ). Hence,

image(f ) is closed.

• image(f) is bounded: assume by contradiction that there is a sequence

(yn ) ⊆ image(f ) such that n ≤ yn for every n. As in the previous point
there is a sequence (xn ) in S with f (xn ) = yn , and since S is compact we
have that there is a convergent subsequence xnk that converges to x ∈ S.
But this is a contradiction, because f (x) ∈ R and:

∞ = limk→∞ ynk = limk→∞ f (xnk ) = f (x) < ∞

All in all, we get that image(f ) is compact, and therefore f obtains a

maximum.
(b) Let v be a vector in S on which f is maximal, we will show that for every
⊥ T
u ∈ span({v})
 ,u  Av = 0. As the hint suggests, define a function g : R → R
v+tu
by g(t) = f ||v+tu|| . By definition of f , we have that for every t ∈ R:

(v+tu)T A(v+tu) v T Av+tuT Av+tv T Au+t2 uT Au

g(t) = ||v+tu||2 = ||v||2 +2thv,ui+t2 ||u||2
0 0
Now, using ( yz )0 = y z−z
z2
y
. Define y(t) = v T Av + tuT Av + tv T Au + t2 uT Au,
and z(t) = ||v|| + 2thv, ui + t2 ||u||2 . We get that
2

limt→0 z 0 (t) = 2hv, ui = 0.2 And limt→0 y 0 (t) = uT Av + v T Au

Therefore,
2u ∈ span({v})⊥

5
 T T
y 0 z−z 0 y
limt→0 g 0 (t) = limt→0 z2 = (u Av+v Au) limt→0 z(t)−0·limt→0 y
limt→0 z 2 (t) =
(uT Av+v T Au)||v||2
||v||4

Now, since f (x) obtains maximum at v. We get that g(t) has a critical point
at t = 0. Therefore, limt→0 g 0 (t) has to be 0, that is, uT Av + v T Au = 0. This
happens iff

hu, Avi = −hv, Aui

And since A is symmetric, this happens iff
hu, Avi = −hAv, ui

Meaning that uT Av = hu, Avi must be 0.

(c) Let v2 , ..., vn ∈ Rn be an orthonormal basis for span({v})⊥ . Then v, v2 , ..., vn
is an orthonormal basis for Rn . We want to show that for every 2 ≤ i ≤ n,
(vi )T Av = 0. This is sufficient, because every vector 0 6= u ∈ Rn can be written
as α1 v + α2 v2 + ... + αn vn , therefore, for every 2 ≤ i ≤ n, we have

(vi )T u = hvi , α1 v + α2 v2 + ... + αn vn i = αi ||vi ||2 = αi .

Meaning that u ∈ span({v}) iff (vi )T u = 0 for every 2 ≤ i ≤ n. This implies
that Av ∈ span({v}), that is, v is an eigenvector of A. What is left to show is
that for every 2 ≤ i ≤ n, (vi )T Av = 0. But this follows immediately from (b),
because every vi , 2 ≤ i ≤ n is in span({v})⊥ .
(d) The spectral theorem can be proven by inductively as follows: first, we
find an eigenvector v of A as in ((a), (b), (c)). Now, consider span({v})⊥ , we
claim that we can inductively find another vector, v2 , in span({v})⊥ which is
also an eigenvector of A. (clearly, this vector is orthogonal to v). We find
v2 as follows: the set S is now modified to be S2 = {x ∈ Rn : ||x|| = 1} ∩
{x ∈ Rn : x ∈ span({v})⊥ }, note that this intersection is isomorphic to the
unit circle in Rn−1 . That is, S2 is compact, and thus we can use the same
proof as in (a), (b), (c) to get that there is an eigenvector v2 ∈ span({v})⊥ .
Now, we can keep applying the same argument, where at iteration i: we define
Si = {x ∈ Rn : ||x|| = 1} ∩ {x ∈ Rn : x ∈ span({v, v1 , ..., vi−1 })⊥ }. Note, that
after we get the n’th vector we stop.