Professional Documents
Culture Documents
Problem 1
(a) Recall that the null space and the range of a matrix A Cmn are defined
as:
N (A) = {x Cn |Ax = 0}
R(A) = {y Cm |y = Ax, for some x Cn }
1
(b) Note that:
x R (A) x Ay = 0, y Cn
y A x = 0, y Cn
A x = 0
x N (A )
(c) N (A) = R(A ) follows from part (b) and the following linear algebra facts.
Let V and W be subspaces of an inner product space (X, <, >); we have:
(V ) = V , for all V .
V = W V = W .
Problem 2
kAxk1
kAk1 = sup = max kAxk1
x6=0 kxk1 kxk1 =1
2
m
X
We begin by showing that kAk1 max |aij |. We have:
1jn
i=1
m X
n
X
kAxk1 =
a x
ij j
i=1 j=1
m X
X n
|aij xj |
i=1 j=1
Xm X n
= |aij ||xj |
i=1 j=1
Xn X m
= |aij ||xj |
j=1 i=1
n m
!
X X
= |xj | |aij |
j=1 i=1
n m
!
X X
|xj | max |aij |
1jn
j=1 i=1
m
X X
= max |aij | |xj |
1jn
i=1 1jn
m
X
= max |aij | kxk1
1jn
i=1
m
X
To show that this upper bound can be achieved, let jo = argmax |aij |, and
j
i=1
= ejo , the joth basis vector (i.e. x
consider x i = 1 when i = jo and 0 otherwise).
Clearly, k
x1 k1 = 1 and:
a1jo m
.. X
Ax = . kAxk1 = |aijo |
amjo i=1
kAxk
kAk = sup = max kAxk
x6=0 kxk kxk =1
3
n
X
We begin by showing that kAk max |aij |. We have:
1im
j=1
n
X
kAxk = max | aij xj |
1im
j=1
n
X
max |aij xj |
1im
j=1
Xn
= max |aij ||xj |
1im
j=1
n
X
max |aij | ( max |xj |)
1im 1jn
j=1
n
X
= max |aij | kxk
1im
j=1
n
X
= max |aij | kxk
1im
j=1
n
X
= max |aij | whenever kxk = 1
1im
j=1
n
X
To show that this upper bound can be achieved, let io = argmax |aij | and
i
j=1
consider vector x
defined as:
sgn(aio 1 )
x
=
..
.
sign(aio n )
Clearly k
xk = 1 since |
xj | = 1 for all j, and
n
X n
X n
X
kA
xk = aio j sgn(aio j ) = |aio j | = max |aij |
1im
j=1 j=1 j=1
4
and
m
X m
X 2
kAk22 = |ai |2 max |ai | = mkAk2
1im
i=1 i=1
When x 6= 0, we have:
kAxk kAxk 1 kAxk2
kxk kxk2 m kxk2
with each of the inequalities following directly from the established vector in-
equalities. Hence:
kAxk 1 kAxk2
kAk = sup for all x 6= 0
x6=0 kxk m kxk2
from which it follows that:
1 kAxk2 1
kAk sup = kAxk2
x6=0 m kxk2 m
which proves the right hand side inequality.
When x 6= 0, it also follows from the established vector identities that
kAxk kAxk2 . Hence we have:
kAxk kAxk2 kAxk2
n n sup n = nkAk2
kxk2 kxk2 x6=0 kxk2
1
Noting that for x Cn the established vector inequality implies that
kxk
n
, we have:
kxk2
kAxk kAxk kAxk
n nkAk2 kAk = sup n nkAk2
kxk kxk2 x6=0 kxk
kAxk22 = kU V xk22
= kV xk22 (Since U is unitary)
= kyk22 where y = V x
Xn
= |i yi |2
i=1
min (A)2 kyk22
= min (A)2 kV xk22
= min (A)2 kxk22 (Since V is unitary)
5
kAxk2
Hence min (A) for all x 6= 0 and min kAxk2 min (A). To show
kxk2 kxk2 =1
that this lower bound can be achieved, pick x = vn , where vn is the nth column
of V . Then:
v1
0
.. ..
V x = . vn = . = en ,
vn1 0
vn 1
V x = en = min (A)em
where em is the basis vector with unity entry in row m, and
Ax = U V x = U min (A)em = min (A)um
where um is the mth column of U . Hence:
kAxk2 = kmin (A)um k2 = min (A)kum k2 = min (A).
When rank(A) < n, note that Ax = 0 for x = vn since V x = 0. Hence
min kAxk2 = 0 in this case.
kxk2 =1
Problem 3
i=k+1 i=k+1
and
kxk22
= kk+1
vk+1 + . . . + n vn k22
= (k+1 vk+1 + . . . + n vn )(k+1 vk+1 + . . . + n vn )
Xn
= |i |2
i=k+1
6
Hence kAxk2 k+1 kxk2 .
(b) We have:
A is invertible det(A) 6= 0
det(U V ) 6= 0
det(U )det()det(V ) 6= 0
det() 6= 0
(A) 6= 0
(A) > 0
where the third equivalence follows because U , and V are square matrices
of identical sizes, and where the fourth equivalence follows because both U and
V are unitary hence invertible.
Note that when A is invertible, A1 is given by
A1 = V 1 U
where
1
0 ... 0
(A)
.. ..
0 . .
1 =
.. ..
. . 0
1
0 ... 0 (A)
Hence, the singular values of A1 are the reciprocals of the singular values of A
with
1
(A1 ) = .
(A)
(c) Recall that a matrix is symmetric positive semidefinite iff all its eigen-
values are non-negative, and that the singular values of a Hermitian matrix are
identical to its eigenvalues. Also, note that the singular value decomposition of
a Hermitian matrix is of the form U U . We have:
(A)I A = (A)IU U U U = U ((A)I ) U
The eigenvalues of (A)I A are then given by 0, (A) 2 (A),..., (A) (A)
and are thus all non-negative, which proves that:
(A)I A 0 A (A)I
Similarly we have:
A + (A)I = U ( + I) U
and the eigenvalues of A + (A)I are clearly non-negative, each being the sum
of two non-negative terms. Hence:
A + (A)I 0 (A) A.
7
Problem 4
sup |u(t)|
is not a norm as it doesnt satisfy the positivity condition:
t
Consider a constant signal u(t) = c > 0, and note that u(t)
= 0 for all t,
hence supt |u(t)|
= 0 even though u(t) is a non-zero signal.
(i) (Positivity) Being the sum of nonnegative quantities, kuk 0 for all
u. Moreover
|u(0)| = 0
kuk = 0 u(t) = 0, for all t 0
|u(t)|
= 0 t 0
(i) (Positivity) Being the sum of nonnegative quantities, kuk 0 for all
u. Moreover
supt |u(t)| = 0
kuk = 0 u(t) = 0, for all t 0
supt |u(t)|
=0
8
what is left to note is that:
(i) (Positivity) Being the sum of nonnegative quantities, kuk 0 for all
u. Moreover
supt |u(t)| = 0
kuk = 0 u(t) = 0, for all t 0
supt |u(t)|
=0