Linear Algebra II

Linear algebra is the branch of mathematics concerned with the study of vectors, vector spaces (also called linear spaces), linear transformations and system of linear equations. Vector spaces is a central theme in modern mathematics; thus, linear algebra is widely used in both abstract algebra and functional analysis. Linear algebra also has a concrete representation in analytic geometry and it is generalized to operator theory. It has extensive application in the natural sciences and the social sciences, since non-linear models can often be approximated by a linear model.

References
1. H. Anton, Elementary Linear Algebra, 7th ed., John Wiley & Sons Inc., 1992. 2. L. W. Johnson, R. D. Riers & J. T. Arnold, Introduction to Linear Algebra, 3rd ed., Addison-Wesley Publ. Co., 1993. 3. D. C. Lay, Linear Algebra and its Applications, 2nd ed., Addison-Wesley Publ. Co., 1996. 4. H. Anton, R. C. Busby, Comtemporary Linear Algebra, John Wiley & Sons Inc., 2003.

Contents
1 Vector Spaces 1.1 1.2 1.3 1.4 1.5 1.6 1.7 Systems of numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vector spaces over a eld . . . . . . . . . . . . . . . . . . . . . . . . . . . . Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Null space of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 2 6 9

Sum of two subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Direct sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 15

2 Bases and Dimension 2.1 2.2 2.3 2.4 2.5

Linear combination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Linear independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Basis and dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Dimension of subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Ordered basis and coordinates . . . . . . . . . . . . . . . . . . . . . . . . . 40 44

3 Linear Transformations 3.1 3.2 3.3 3.4 3.5 3.6

Linear transformation (mappings, operators) . . . . . . . . . . . . . . . . . 44 Properties of linear mappings . . . . . . . . . . . . . . . . . . . . . . . . . 46 Linear mappings from Mn,1 (F ) to Mm,1 (F ) . . . . . . . . . . . . . . . . . . 46 Range and null space of a linear map . . . . . . . . . . . . . . . . . . . . . 51 Matrices associated with linear transformations . . . . . . . . . . . . . . . 60 Properties of similar matrices . . . . . . . . . . . . . . . . . . . . . . . . . 63 65

4 Eigenvalues and Eigenvectors 4.1 4.2

Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Matrix polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Chapter 1 Vector Spaces

1.1 Systems of numbers

(i) The set of all integers Z = {. . . , 2, 1, 0, 1, 2, . . . }. (ii) The set of all positive integers N = {1, 2, . . . }. : p, q Z, q = 0}. (iii) The set of all rational numbers Q = { p q (iv) R denotes the set of all real numbers. (v) The set of all complex numbers C = {a + bi : a, b R}, i2 = 1. For each of the above sets, there are two operations: addition + and multiplication satisfying some laws such that associative laws: a + (b + c) = (a + b) + c, a (b c) = (a b) c and distributive laws: a (b + c) = a b + a c, (a + b) c = a c + b c.

1.2

Rings

Rings is an algebraic system which generalizes many systems of numbers like Z, Q, R, C. Denition 1.2.1 A non-empty set R with two operations called addition + and multiplication , is said to be a ring if a, b, c R, (i) a + b and a b are unique elements in R; (ii) a + (b + c) = (a + b) + c, a (b c) = (a b) c (associative laws); (iii) a + b = b + a (commutative law); (iv) there exists a unique element in R, called zero and denoted by 0 such that a + 0 = 0 + a = a (existence of identity with respect to addition); 0

(v) there exists a unique element in R denoted by a such that a + (a) = (a) + a = 0 (existence of inverse with respect to addition); (vi) a (b + c) = a b + a c, (a + b) c = a c + b c (distributive laws). A ring R is called a eld if in addition to the above axiom, (vii) a b = b a a, b R (commutative law with respect to multiplication); (viii) there exists a unique element in R called an identity and denoted by 1 such that 1 a = a 1 = a, a R (existence of identity with respect to multiplication); (ix) If a = 0, then there exists a unique element a1 R such that a1 a = a a1 = 1 (a1 is called the multiplicative inverse of a). Example 1.2.1 Let + and denote the usual addition and multiplication for numbers. Then Q, R and C are all elds with respect to + and . Z is a ring which is not a eld since 2 Z \ {0}, but there does not exist d Z such that 2 d = 1. Example 1.2.2 Let F = {a, b, c} with the following 2 operations: + a a a b b c c Then (F, +, ) is a eld. b c b c c a a b a b c a a a a b c a a b c c b

1.3

Vector spaces over a eld

Let F be a eld and V be a non-empty set with 2 operations: addition + and scalar multiplication such that for any u, v, w V and a, b F : A1. u + v is a unique vector in V ; A2. (u + v ) + w = u + (v + w ) (associative law); A3. u + v = v + u (commutative laws); A4. There exists an element 0 in V , called the zero vector such that u + 0 = 0 + u = u; A5. For each u V , there exists u V , called the additive inverse of u such that u + u = 0 = u + (u); 1

B1. a u is a unique element in V ; B2. a (u + v ) = a u + a v ; B3. (a + b) u = a u + b u; B4. a (b u) = (a b) u; B5. 1 u = u. Then V is called a vector space over the eld F . Elements in V are called vectors and elements in F are called scalars. Denition 1.3.1 Let A be a non-empty set. Then An := {(a1 , . . . , an ) : ai A, i = 1, . . . , n} is called the n-fold Cartesian product of A. Theorem 1.3.1 For each positive integer n, Rn is a vector space over R with respect to the following 2 operations: (a1 , . . . , an ) + (b1 , . . . , bn ) = (a1 + b1 , . . . , an + bn ) k (a1 , . . . , an ) = (ka1 , . . . , kan ) where ai , bi R, i = 1, . . . , n and k R.

Proof. Let u = (a1 , . . . , an ), v = (b1 , . . . , bn ), w = (c1 , . . . , cn ) be 3 elements in Rn and k, l R. A1. u + v = (a1 + b1 , . . . , an + bn ) Rn since ai + bi R i = 1, . . . , n. A2.(u + v ) + w = (a1 + b1 , . . . , an + bn ) + (c1 , . . . , cn ) = ((a1 + b1 ) + c1 , . . . , (an + bn ) + cn ) u + (v + w ) = (a1 , . . . , an ) + (b1 + c1 , . . . , bn + cn ) = (a1 + (b1 + c1 ), . . . , an + (bn + cn )) Since ai + (bi + ci ) = (ai + bi ) + ci i = {1, . . . , n}, it follows that (u + v ) + w = u + (v + w ). A3. u + v = (a1 + b1 , . . . , an + bn ) v + u = (b1 + a1 , . . . , an + bn ) Since ai + bi = bi + ai i, it follows that u + v = v + u. A4. Let 0 = (0, . . . , 0) Rn . Then 0 + u = (0 + a1 , . . . , 0 + an ) = (a1 , . . . , an ) = u u + 0 = (a1 + 0, . . . , an + 0) = (a1 , . . . , an ) = u. 2

A5. Let u = (a1 , . . . , an ). Then u + (u) = (a1 , . . . , an ) + (a1 , . . . , an ) = (a1 + (a1 ), . . . , an + (an )) = (0, . . . , 0) = 0 u + u = 0 (From A3.) B1. k u = (ka1 , . . . , kan ) Rn since kai R i = 1, . . . , n. B2. k (u + v ) = k (a1 + b1 , . . . , an + bn ) = (k (a1 + b1 ), . . . , k (an + bn )) = (ka1 + kb1 , . . . , kan + kbn ) = (ka1 , . . . , kan ) + (kb1 , . . . , kbn ) = k u + k v. B3. (k + l) u = (k + l) (a1 , . . . , an ) = ((k + l)a1 , . . . , (k + l)an ) k u + l u = k (a1 , . . . , an ) + l (a1 , . . . , an ) = (ka1 , . . . , kan ) + (la1 , . . . , lan ) = (ka1 + la1 , . . . , kan + lan ) Since (k + l)ai = kai + lai i, it follows that (k + l) u = k u + l u. B4. (kl) u = (kl) (a1 , . . . , an ) = ((kl)a1 , . . . , (kl)an ) k (l u) = k (la1 , . . . , lan ) = (k (la1 ), . . . , k (lan )) Since (kl)ai = k (lai ) i, we have (kl) u = k (l u). B5. 1 u = 1 (a1 , . . . , an ) = (1a1 , . . . , 1an ) = (a1 , . . . , an ) = u Rn is a vector space over R. Theorem 1.3.2 Let F be a eld. Let Mm,n (F ) denote the set of all m n matrices over F , i.e., Mm,n (F ) = {(aij ) : aij F, i = 1, . . . , m, j = 1, . . . , n}. Then Mm,n (F ) is a vector space over F under the following operations: (aij ) + (bij ) = (aij + bij ); (aij ) = (aij ), F, aij , bij Mm,n (F ).

(Mm,n (F ) is called the vector space of m n matrices over F .) Let 0 denote the m n matrix with all entries equal to 0 F . Then 0 + A = A + 0 = A A Mm,n (F ). For each A = (aij ) Mm,n (F ), let A = (aij ). Then A + (A) = (A) + A = 0. Remark 1.3.1 Theorem 1.3.1 is a special case of Theorem 1.3.2 since . 3

Example 1.3.1 (Vector space of real functions) Let F (, ) denote the set of all functions f : R R. Let f, g F (, ) and c R. Dene the addition and scalar multiplication operations as follows: (f + g ) (x) = f (x) + g (x) x R (cf ) (x) = c (f (x)) x R. Then F (, ) is a vector space over the eld R. (Exercise) Let 0 : R R be the function dened by 0(x) = 0 x R. Then 0 + f = f + 0 = f f F (, ). 0 is called the zero function and is the zero vector in F (, ). For each f F (, ), dene f as follows: (f )(x) = f (x) x R. Then (f ) + f = 0 = f + (f ). Example 1.3.2 (Vector space of all real polynomials) Let R[x] denote the set of all real polynomials, i.e., R[x] = {a0 + a1 x + + an xn : n Z+ {0}, ai R, i = 1, . . . , n}. For any f, g R[x], c R dene two operations as follows: (f + g ) (x) = f (x) + g (x) x R; (cf ) (x) = c (f (x)) x R. Then R[x] is a vector space over R. (Exercise) Denition 1.3.2 (Substraction of vectors) Let u and v be two vectors in a vector space V . Then u v is dened as the vector u + (v ). (Note that u u = u + (u) = 0 u V .) Property 1: Let u, v, w be three vectors in a vector space V . (a) If u + v = w , then u = w v ; (b) If u + v = 0, then u = v . (Exercise) Proof of (a): From u + v = w , we have (u + v ) + (v ) = w + (v ) u + (v + (v )) = w v u+0=wv u = w v. 4 ( v + (v ) = 0)

Theorem 1.3.3 Let V be a vector space over a eld F . Then for any u V and c F , we have (i) c 0 = 0; (ii) 0 u = 0; Proof. (i) c 0 = c (0 + 0) = c 0 + c 0 (Axiom B2) c0=c0c0=0 (Property 1(a)) (iii) c u = 0 c = 0 or u = 0; (iv) (c) u = c (u) = (c u).

(ii) 0 u = (0 + 0) u = 0 u + 0 u (Axiom B3) 0 u = 0 u 0 u = 0. (iii) Suppose that c u = 0. If c = 0, we are done. If c = 0, then c1 F such that c1 c = 1. c u = 0 c1 (c u) = c1 0 = 0 (By (i) and (c1 c) u = 1 u = u (B5)). Hence u = 0.

1.4

Subspaces

Denition 1.4.1 Let V be a vector space over a eld F . Then a non-empty subset U of V is called a subspace of V if under the operations of V , U itself, forms a vector space over F . We write U V if U is a subspace of V . Example 1.4.1 In R[x], let Vn be the set of all polynomials of degree < n where n is a xed positive integer. Then Vn is a vector space over R under the usual operations for polynomials of addition and scalar multiplication. Thus Vn is a subspace of R[x]. Note that R[x] is a subspace of the vector space F (, ) of real functions. Theorem 1.4.1 Let V be a vector space over a eld F and = U V . Then U is a subspace of V if and only if (i) u + v U u, v U ; 5 (ii) cu U u U, c F .

Proof. () Suppose that U V . Then vectors in U satisfy Axiom A1 and B1. Hence conditions (i) and (ii) are true. () Suppose that conditions (i) and (ii) are true. We wish to show that U V . Note that vectors in V satisfying A2, A3 and B2B5 and U V , it follows that vectors in U satisfying A2, A3 and B2B5. A1 is satised by U because of condition (i). B1 is satised by U because of condition (ii). From condition (ii), we have 0u = 0 U . Note that 0 + u = u+ 0 = u u U A4 is satised. Let u U . From condition (ii), (1)u = (1u) = u U . Since u + (u) = (u) + u = 0. A5 is satised. Hence U is a vector space over F , i.e., U V . since U V .

Theorem 1.4.2 Let V be a vector space over a eld F and U V . Then U V if and only if (i) U = ; (ii) cu + dv U c, d F and u, v U .

Proof. () Suppose U V . Then by denition U = . Let c, d F and u, v U . From condition (ii) of Theorem 1.4.1, we have cu U and dv U . Again from the condition (i) of Theorem 1.4.1, cu + dv U ( U V ). () Let u, v U and c F . By hypothesis (ii), cu + 0v U . Hence cu U since 0v = 0. By hypothesis (ii), 1u + 1v U . Hence u + v U ( 1u = u, 1v = v ). By Theorem 1.4.1, we conclude that U V .

Example 1.4.2 Let U = {(x, y ) : y = x + 1}. U is not a subspace of R2 since (0, 0) / U. y 1 1 x

Example 1.4.3 Let U = {(x, y ) : x, y R, y = 2x}. Clearly U = ((0, 0)) U ). Let u, v U and c R. Then u = (x, 2x), v = (z, 2z ) for some x, z R u + v = (x, 2x) + (z, 2z ) = (x + z, 2(x + z )) U cu = c(x, 2x) = (cx, 2(cx)) U. By Theorem 1.4.1, U R2 . Example 1.4.4 Let V be a vector space over a eld F . Then {0} V and V V.

Example 1.4.5 Let U = {(x, y, z ) R3 : x = y }. Then (0, 0, 0) U . Let u, v U , R. Then u = (a, a, b), v = (c, c, d) for some a, b, c, d R. (i) u + v = (a, a, b) + (c, c, d) = (a + c, a + c, b + d) U ; (ii) u = (a, a, b) = (a, a, b) U . It follows from Theorem 1.4.1 that U R3 . (Note that U is a plane passing through the origin.)

Example 1.4.6 Let U = of M2 (R) since 1 0 0 0

a b c d U , but (1)

: a 0, a, b, c, d R . Then U is not a subspace

1 0 0 0

1 0 0 0

/ U.

Example 1.4.7 Let C n (, ) be the set of all real functions from R to R which have continuous nth derivatives. It can be shown that C n (, ) F (, ). Exercise: Let A be the set of all functions in C 2 (, ) such that f (x) + f (x) = 0 x R. Prove that A C 2 (, ). 7

1.5

Null space of a matrix

Let A Mm,n (F ) where F is a eld. Let N (A) = {u Mn,1 (F ) : Au = 0m,1 }. (N (A) is the solution set of the linear system Ax = 0m,1 .) Theorem 1.5.1 N (A) is a subspace of Mn,1 (F ). Proof. Clearly A0 = 0m,1 where 0 Mn,1 (F ). N (A) = . Let u, v N (A) and F . (i) Au = 0m,1 , Av = 0m,1 A(u + v ) = Au + Av = 0m,1 + 0m,1 = 0m,1 u + v = N (A). (ii) A(u) = (Au) = 0m,1 = 0m,1 . u N (A). N (A) Mn,1 (F ). Denition 1.5.1 N (A) is called the null space of A. 1 1 1 0 2 1 0 1

Example 1.5.1 Determine N (A) if A = Solution: 1 1 1 0 2 1 0 1

1 2

M2,4 (R).

0 0 0 0

2 1

R 2R

1 1 1 0 0 1 2 1 1 0 1 1 0 1 2 1

0 0 0 0

R +R

1 0 1 1 0 1 2 1

2 2

R R

which is a reduced row echelon form. x1 x3 + x4 = 0 x2 + 2x3 x4 = 0 x1 = x3 x4 x2 = 2x3 + x4 zw 2z + w : z, w R . z w

There are 2 free variables x3 , x4 . Hence, N (A) = Note that every vector in N (A) is of the 1 2 z 1 +w 0 form 1 1 0 1 8

where z, w R.

Theorem 1.5.2 If U and V are subspaces of a vector space W over a eld F , then U V W. Proof. (i) U W 0 U ; V W 0 V . 0 U V, U V = .

(ii) Let u, v U V , F . Then u, v U and u, v V . Since U, V W , it follows that u + v U , u U and u + v V , u V (by Theorem 1.4.1). Hence u + v U V and u U V . U V W .

Example 1.5.2 Let U = {(x, y, 0) : x, y R} (xy -plane), V = {(0, y, z ) : y, z R} (yz -plane). Then it can be shown that U R3 , V R3 . Note that U V = {(0, y, 0) : y R} (y -axis) R3 .

Example 1.5.3 Let S1 = {(x, y, z ) R3 : x + 2y + 3z = 0}, S2 = {(x, y, z ) R3 : 2x + 3y + 4z = 0}. It can be shown that S1 R3 and S2 R3 . (S1 , S2 are two planes in R3 .) What is S1 S2 ?

1.6

Sum of two subspaces

Theorem 1.6.1 Let U W and V W . Then U + V := {u + v : u U, v V } W . (U + V is called the sum of U and V .) Proof. (i) Note that 0 U , 0 V and hence 0 + 0 U + V , that is, 0 U + V . U + V = . (ii) Let x, y U + V and F . Then x = u1 + v1 y = u2 + v2 for some u1 U, v1 V for some u2 U, v2 V . 9

Hence, x + y = (u1 + v1 ) + (u2 + v2 ) = (u1 + u2 ) + (v1 + v2 ) (from Axiom A2 , A3 ). Since u1 , u2 U, v1 , v2 V, U W, V W, we have u1 + u2 U and we have v1 + v2 V . (1.1) (1.2)

(u1 + u2 ) + (v1 + v2 ) U + V, that is, x + y U + V.

(from (1.1), (1.2) and the def. of the sum)

(iii) x = (u1 + v1 ) = u1 + v1 (Axiom B2 ). Since u1 U, v1 V, F F and U W, and V W, we have u1 U and we have v1 V

x = u1 + v1 U + V. It follows from Theorem 1.4.1 that U + V W . Remark 1.6.1 If U W, V W , then U U + V and V U + V . (For any u U , u = u + 0 U + V since 0 V, u U , U U + V and for any v U , v = 0 + v U + V since 0 U, v V , V U + V . )

U V U U +V W U V V U + V W. W U U V V

U +V

Example 1.6.1 Let L1 = {(x, y ) : y = 2x, x, y R}, L2 = {(x, y ) : y = 4x, x, y R}. Then L1 , L2 R2 , L1 , L2 are straight lines passing through the origin. What is L1 + L2 ? 10

Solution: Let (a, b) R2 . Then (a, b) L1 + L2 (a, b) = (x1 , 2x1 ) + (x2 , 4x2 ) for some x1 , x2 R a = x1 + x2 b = 2x1 4x2 . 1 1 2 4 a b
2 1

(1.3)

R 2R

1 1 0 6

a b 2a
b+4a , 6

1 2

6R +R

6 0 0 6

b + 4a b 2a

System (1.3) has a unique solution x1 = L1 + L2 =

x2 =

2ab . 6

Remark 1.6.2 Let L1 , L2 be 2 distinct straight lines in R3 passing through the origin (0, 0, 0). Then L1 R3 , L2 R3 and L1 + L2 is the plane determined by L1 and L2 .

1.7

Direct sums

Let L = {(0, 0, z ) : z R} and P = {(x, y, 0) : x, y R}. Then L is the z -axis and P is the xy -plane. Note that L R3 and P R3 , L P = {0} and R3 = L + P . In this case we say that R3 is the direct sum of L and P . Denition 1.7.1 Let U and V be subspaces of a vector space W over a eld F . Then W is called the direct sum of U and V if (i) W = U + V ; In this case we write W = U V . (ii) U V = {0}.

Example 1.7.1 Let L = {(z, z, z ) : z R} and P = {(x, y, 0) : x, y R}. Then L is a straight line passing through the origin and P is the xy -plane. Is R3 = L P ? Solution: (i) We rst show that R3 = L + P . Let (a, b, c) R3 . Then (a, b, c) L + P (a, b, c) = (z, z, z ) + (x, y, 0) for some x, y, z R a = x + z, b = y + z, c = z z = c, y = b z = b c, x = a z = a c. (a, b, c) = (c, c, c) + (a c, b c, 0) L + P R3 L + P. 11

Clearly L + P R3 . L + P = R3 . (ii) (a, b, c) L P (a, b, c) L and (a, b, c) P a = b = c and c = 0 a = b = c = 0. L P = {(0, 0, 0)} Hence R3 = L P . Remark 1.7.1 In general, if P is a plane in R3 passing through the origin, L is a straight line passing through the origin and L W
z z

P , then it can be shown that R3 = L P . W W

z z

u+v v u

z z

ku u

ku u

y
y y

y
y

W is closed under addition z u v

x
x

W is closed under scalar multiplication

W is closed under scalar multiplication

C u y O v A

B w

x The vectors u + v and ku both lie in the same plane as u and v Remark 1.7.2 Subspaces of R2 {0} R2 {0}

Subspaces of R3

Straight lines passing through the origin Straight lines passing through the origin Plane passing through the origin R3 12

F (, ) C (, ) C (, ) C n (, ) R[x]

Remark 1.7.3 Let U = V = W = K=

a 0 0

Then U V M3 (R), M3 (R) = W K .

0 b 0 : a, b, c R 0 0 c a d e 0 b f : a, b, c, d, e, f R 0 0 c a d e d b f : a, b, c, d, e, f R e f c 0 a b a 0 c : a, b, c R b c 0

(diagonal matrices)

(upper triangular matrices)

(symmetric matrices)

(skew-symmetric matrices)

Question: Let P1 = {(x, y, z ) R3 : x + 2y z = 0}, P2 = {(x, y, z ) R3 : 2x y + 4z = 0} be two planes. Is it true that (i) R3 = P1 + P2 , (ii) P1 P2 = {(0, 0, 0)}, (iii) R3 = P1 P2 ?

13

Chapter 2 Bases and Dimension

2.1 Linear combination

Denition 2.1.1 Let V be a vector space over a eld F . A vector v V is called a linear combination of vectors v1 , . . . , vk in V if there exist scalars 1 , . . . , k F such that v = 1 v1 + + k vk . Example 2.1.1 Every vector (a, b, c) R3 is a linear combination of the vector i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1) since (a, b, c) = ai + bj + ck. Example 2.1.2 Determine whether w = (9, 2, 7) is a linear combination of u = (1, 2, 1), v = (6, 4, 2). Solution: w is a linear combination of u, v if and only if c1 , c2 R such that (9, 2, 7) = c1 (1, 2, 1) + c2 (6, 4, 2) = (c1 + 6c2 , 2c1 + 4c2 , c1 + 2c2 ). Consider the system of equations: c1 + 6 c2 = 9 , 2 c1 + 4 c2 = 2 , c1 + 2c2 = 7. 1 6 9 1 6 9 1 6 9 1 6 9

2 4 1 2

We obtain c1 = 3, c2 = 2 and hence w = 3u + 2v . (w is a linear combination of u, v .) 14

R2 2R1 0 8 2 R3 +R1 7 0 8

R3 +R2 16 0 8 16 0 0

1 R 8 2 0 1 16 0 0 0

2 0

Example 2.1.3 Determine whether w = (4, 1, 8) R3 is a linear combination of u = (1, 2, 1) and v = (6, 4, 2) R3 . Solution: w is a linear combination of u, v if and only if c1 , c2 R such that (4, 1, 8) = c1 (1, 2, 1) + c2 (6, 4, 2) = (c1 + 6c2 , 2c1 + 4c2 , c1 + 2c2 ). Consider the system of equations: c1 + 6 c2 = 4 , 2c1 + 4c2 = 1, c1 + 2c2 = 8. 1 6 4 1 6 4 1 6 4 (2.1)

The system (2.1) is inconsistent. w is not a linear combination of u and v .

2 4 1 2

R2 2R1 0 8 1 R3 +R1 8 0 8

R3 +R2 9 0 8 12 0 0

9 3

Notation: Let u1 , . . . , uk be vectors in a vector space U over a eld F . Then u1 , . . . , uk denotes the set of all linear combinations of u1 , . . . , uk , i.e.,
k

u1 , . . . , uk =
i=1

ai ui : ai F, i = 1, . . . , k .

Theorem 2.1.1 u1 , . . . , uk U . Proof. (i) 0u1 + + 0uk = 0 u1 , . . . , uk . (ii) Let x, y u1 , . . . , uk and F . Then x = a1 u1 + + ak uk y = b1 u1 + + bk uk Then x + y = (a1 u1 + + ak uk ) + (b1 u1 + + bk uk ) = (a1 u1 + b1 u1 ) + + (ai ui + bi ui ) + + (ak uk + bk uk ) = (a1 + b1 )u1 + + (ai + bi )ui + + (ak + bk )uk u1 , . . . , uk . 15 for some ai , bi F, i = 1, . . . , k .

(iii)

x = (a1 u1 + + ak uk ) = (a1 u1 ) + + (ak uk ) = (a1 )u1 + + (ak )uk u1 , . . . , uk . Axiom (B2) Axiom (B4)

From Theorem 1.4.1, we have u1 , . . . , uk U . Denition 2.1.2 u1 , . . . , uk is called the subspace of U generated (spanned) by u1 , . . . , uk . Notation: If S = {u1, . . . , uk }, then we use S to denote u1 , . . . , uk . Remark 2.1.1 (i) u1, . . . , us , . . . , uk = u1, . . . , us + us+1, . . . , uk . (ii) u = {u : F }. (iii) u1, u2 = {a1 u1 + a2 u2 : a1 , a2 F }. (iv) u1 u1 , u2 , u3 u2 u1 , u2 , u3 u3 u1 , u2 , u3 since u1 = 1u1 + 0u2 + 0u3 since u2 = 0u1 + 1u2 + 0u3 . since u3 = 0u1 + 0u2 + 1u3

{u1, u2 , u3 } u1 , u2 , u3 . In general, u 1 = 1u 1 + 0u 2 + + 0u k u 1 , . . . , u k ; u i = 1 u 1 + + i u i + + k u k , u k = 0u 1 + 0u 2 + + 0u k 1 + 1u k . {u 1 , . . . , u k } u 1 , . . . , u k . Remark 2.1.2 (i) If u R3 \{0}, then u is the line through the origin determined by u. (ii) If u, v R3 \{0} and u / v , then u, v is the plane through the origin determined by u and v . 16 where i = 1, j = 0 j = i;

Span [u]
z z

z k1 u

Span [u, v ] k1 u + k2 v

ku u

k2 v

y
y

Question: When u1 , . . . , uk = v1 , . . . , vs ? Answer: u1 , . . . , uk = v1 , . . . , vs each ui is and each vi is Example 2.1.4 (0, 1, 0), (1, 0, 0) = {x(0, 1, 0) + y (1, 0, 0) : x, y R} = {(y, x, 0) : x, y R} = xy plane (i = 1 , . . . , k ) (i = 1 , . . . , s ).

Example 2.1.5 Let Pn denote the set of all real polynomials of degree n, where n is a positive integer. Then Pn R[x]. Note that Pn = .

Example 2.1.6 R3 is generated by i = (1, 0, 0), j = (0, 1, 0), k = (0, 0, 1). (1, 0, 0), (0, 1, 0), (0, 0, 1) ={a(1, 0, 0) + b(0, 1, 0) + c(0, 0, 1) : a, b, c R} ={(a, b, c) : a, b, c R} = R3 .

Example 2.1.7 Is R3 generated by u1 = (1, 1, 2), u2 = (1, 0, 1) and u3 = (2, 1, 3), i.e., R3 = u 1 , u 2 , u 3 ? 17

x
x

Span [u] is the line through the origin determined by u

x Span [u, v ] is the plane through the origin determined by u and v

Solution (a): For any (a1 , a2 , a3 ) R3 , consider whether (a1 , a2 , a3 ) can be written as a linear combination as follows (a1 , a2 , a3 ) = c1 (1, 1, 2) + c2 (1, 0, 1) + c3 (2, 1, 3) = (c1 + c2 + 2c3 , c1 + c3 , 2c1 + c2 + 3c3 ). We obtain a system of linear equations: c1 + c2 + 2c3 = a1 c1 + + c3 = a2 (2.2)

2c1 + c2 + 3c2 = a3 1 1 2 1 0 1 2 1 3 a1 1 1 2 R2 R1 a2 0 1 1 R3 2R1 a3 0 1 1 a1 a2 a1 a3 2a1 1 1 2 R3 R2 0 1 1 0 0 0 a1 a2 a1 a3 a1 a2

The system (2.2) has a solution if and only if a3 a1 a2 = 0.

Take (a1 , a2 , a3 ) = (1, 1, 1), (2.2) has no solution. Hence (1, 1, 1) / u1 , u2 , u3 . u 1 , u 2 , u 3 = R3 . Solution (b): Note that u3 = u1 + u2 . u u1 , u2 , u3 u = a1 u1 + a2 u2 + a3 (u1 + u2 ) = (a1 + a3 )u1 + (a2 + a3 )u2 u1 , u2, u3 is a plane passing through the origin determined by u1 , u2 . Thus, R3 = u1, u2 , u3 . Example 2.1.8 Is R3 = (1, 1, 1), (1, 1, 0), (1, 0, 0) ? Solution: R3 = (1, 1, 1), (1, 1, 0), (1, 0, 0) if and only if every vector (a, b, c) R3 can be written as a linear combination as follows (a, b, c) = 1 (1, 1, 1) + 2 (1, 1, 0) + 3 (1, 0, 0) for some 1 , 2 , 3 R. Consider the linear system: 1 + 2 + 3 = a 1 + 2 =b (2.3)

1 =c The system (2.3) has a unique solution: 1 = c, 2 = b c, 3 = a b. (a, b, c) = c(1, 1, 1) + (b c)(1, 1, 0) + (a b)(1, 0, 0). R3 = (1, 1, 1), (1, 1, 0), (1, 0, 0) . 18

2.2
Then

Linear independence

Denition 2.2.1 Let v1 , . . . , vk be k vectors in a vector space. Let S = {v1 , v2 , . . . , vk }.

(i) S is called a linearly independent set or v1 , v2 . . . , vk are called linearly independent if the equation c1 v1 + c2 v2 + + ck vk = 0 (ci F, i = 1, . . . , k ) has only the trivial solution c1 = 0, c2 = 0, . . . , ck = 0. (That is, c1 v1 + c2 v2 + + ck vk = 0 c1 = c2 = ck = 0.) (ii) S is called a linearly dependent set or v1 , v2 , . . . , vk are called linearly dependent if there exist scalars c1 , c2 , . . . , ck not all zero such that c1 v1 + c2 v2 + + ck vk = 0 (ci F, i = 1, . . . , k ). Example 2.2.1 Determine whether the vectors u, v, w are linearly independent where (i) u = (1, 1, 1), v = (1, 1, 0), w = (1, 0, 0); (ii) u = (1, 2, 4), v = (2, 1, 3), w = (4, 1, 1). Solution: (i) Consider the equation: 1 (1, 1, 1) + 2 (1, 1, 0) + 3 (1, 0, 0) = (0, 0, 0), i.e., 1 + 2 + 3 = 0 1 + 2 1 =0 =0 (2.4)

Clearly, (2.4) has only the trivial solution 1 = 2 = 3 = 0. u, v, w are linearly independent. (ii) Consider the equation: 1 (1, 2, 4) + 2 (2, 1, 3) + 3 (4, 1, 1) = (0, 0, 0), i.e., 1 + 22 + 43 = 0 2 1 + 2 3 = 0 4 1 + 32 + 3 = 0 1 2 4 0 1 2 4 0 1 2 4 0 (2.5)

2 1 1 4 3 1

R2 2R1 0 3 9 0 R3 4R1 0 0 5 15 19

1 3 R2 0 1 0 1 3 5 R3 0 0 1 3

0 0

 0 2 t . Set of solution of (2.5) is 3t : t R = 0 t 0 u, v, w are linearly dependent.

R3 R2 0 1 3

1 2 4

0 0 0

R1 2R2 0 0 1 0 0 0

1 0 2 3 0

0 . 0

z z

v2 v1 v1
y
y

z z

z z

v1 v2

v2

y
y y

x
x

(a) Linearly dependent

(b) Linearly dependent

(c) Linearly independent

z v1 v2

z v2 v1 y v3 v3 y

v1 v2

y v3

x (a) Linearly dependent (b) Linearly dependent (c) Linearly independent

20

Theorem 2.2.1 k vectors v1 , . . . , vk (k 2) in a vector space V over a eld F are linearly dependent 1 i k such that vi is a linear combination of the other k 1 vectors. Proof. () Suppose that v1 , . . . , vk are linearly dependent. Then c1 v1 + + ck vk = 0 for some scalars c1 , . . . , ck F not all zero. If c1 = 0, then c1 v1 = c2 v2 c3 v3 ck vk and
1 1 c 1 (c1 v1 ) = c1 (c2 v2 c3 v3 ck vk ) 1 1 1 v1 = (c 1 c2 )v2 + (c1 c3 )v3 + + (c1 ck )vk .

v1 is a linear combination of v2 , v3 , . . . , vk . Similarly, if cj = 0 for some j = 2, . . . , k , then vj is a linear combination of the other k 1 vectors. () (Exercise)

Remark 2.2.1 Given vectors v1 , . . . , vs , . . . , vk with vs = 0. Then v1 , . . . , vs , . . . , vk are linearly dependent since 0v1 + 0vs1 + 1vs + 0vs+1 + + 0vk = 0.

Remark 2.2.2 Let v1 , v2 V \{0}, V a vector space. Then v1 , v2 are linearly dependent v2 = v1 for some F .

1 Example 2.2.2 v1 , v2 , 2v1 + 1 v are linearly dependent since 2v1 + 3 v2 is a linear combi3 2

nation of v1 , v2 (Theorem 2.2.1).

Remark 2.2.3 (Remark A) v1 , v2 , v3 v2 , v3 , v1 are linearly independent are linearly independent.

21

Remark 2.2.4 (Remark B) v1 , v2 , v3 v1 , v2 + v3 , v3 are linearly independent are linearly independent (here F ).

Remark 2.2.5 (Remark C) v1 , v2 , v3 v1 , v2 , v3 are linearly independent are linearly independent where F \{0}.

Remark 2.2.6 (Remark D) Let v1 = (a1 , a2 , a3 , a4 ); v2 = (b1 , b2 , b3 , b4 ); v3 = (c1 , c2 , c3 , c4 ); a1 a2 a3 a4 w1 = (d1 , d2 , d3 , d4 ) F 4 w2 = (e1 , e2 , e3 , e4 ) F 4 w3 = (f1 , f2 , f3 , f4 ) F 4 d1 d2 d3 d4

e1 e2 e3 e4 , such that b b b b 1 2 3 4 row equivalent c1 c2 c3 c4 f1 f2 f3 f4 v1 w1 v2 w2 . i.e., row equivalent v3 w3 Then from Remark A, B, C, we see that v1 , v2 , v3 w1 , w2 , w3

are linearly independent are linearly independent.

Remark 2.2.7 (Remark E) Let 0 a1 a2 a3 a4 0 0 b b b 1 2 3 A= 0 0 0 0 c1 0 0 0 0 0

M4,5 (R) 22

where a1 = 0, b1 = 0, c1 = 0.

Then A is a row echelon matrix having 3 nonzero rows u1 = (0, a1 , a2 , a3 , a4 ) u2 = (0, 0, b1 , b2 , b3 ) u3 = (0, 0, 0, 0, c1). Hence, u1 , u2, u3 are linearly independent since 1 u 1 + 2 u 2 + 3 u 3 = 0 (0, 1 a1 , 1 a2 + 2 b1 , 1 a3 + 2 b2 , 1 a4 + 2 b3 + 3 c1 ) = 0 1 a1 = 0, 1 a2 + 2 b1 = 0, 1 a3 + 2 b2 = 0, 1 a4 + 2 b3 + 3 c1 = 0 1 = 0( a1 = 0), 2 = 0( b1 = 0, 1 = 0), 3 = 0( c1 = 0, 1 = 2 = 0). For this example, we see that nonzero rows of this row echelon matrix A are linearly independent. In general, we have Theorem 2.2.2 If E is a row echelon matrix, then the nonzero rows of E are linearly independent.

Theorem 2.2.3 If A is row equivalent to B , then the rows of A are linearly independent if and only if the rows of B are linearly independent.

Example 2.2.3 Determine whether vectors u1, u2 , u3 are linearly independent. (i) u1 = (0, 0, 0, 1), u2 = (1, 0, 3, 4), u2 = (2, 1, 5, 7), u3 =(0, 2, 3, 5); u3 = (3, 2, 8, 11).

(ii) u1 = (1, 0, 2, 3), Solution: (i) A= 1 0

which is a row echelon form. Since u2 , u3 , u1 are nonzero rows of a row echelon matrix, therefore u1 , u2 , u3 are linearly independent.

0 2 3 5 = u3 4 u1 0 2 3 5 0 0 0 1 0 3

u2

23

Since (1, 0, 2, 3), (0, 1, 1, 1), (0, 0, 0, 0) are linearly dependent, therefore u1 , u2 , u3 are linearly dependent. Theorem 2.2.4 If A E , then the rows of A are linearly independent E has no zero rows. (This theorem follows from Theorem 2.2.2 and Theorem 2.2.3.) a11 a12 a13 Theorem 2.2.5 Vectors v1 = a21 , v2 = a22 , v3 = a23 M3,1 (R) are a31 a32 a33 linearly independent if and only if a11 a12 a13 a21 a22 a23 = 0. a31 a32 a33 Proof. v1 , v2 , v3 are linearly dependent 1 , 2 , 3 R, not all a11 a12 a13 0 1 a21 + 2 a22 + 3 a23 = 0 a31 a32 a33 0 i.e., such that zero such that ,

A= 2 1 5

(ii)

1 0 2

R2 2R1 R3 2R2 7 0 1 1 1 0 1 1 1 R3 3R1 3 2 8 11 0 2 2 2 0 0 0 0

1 0 2 3

1 0 2 3

a11 1 + a12 2 + a13 3 = 0 a21 1 + a22 2 + a23 3 = 0 a31 1 + a32 2 + a33 3 = 0 if and only if the system of homogeneous linear equations a11 x1 + a12 x2 + a13 x3 = 0 a21 x1 + a22 x2 + a23 x3 = 0 a31 x1 + a32 x2 + a33 x3 = 0 24

has a nontrivial solution. the matrix (aij ) is not invertible. a11 a12 a13 a21 a22 a23 = 0. a31 a32 a33 a22 a2n , v2 = . , . . . , vn = . Mn,1 (R) are lin. . . . an1 an2 ann early independent |(aij )| = 0. a21 . . . Theorem 2.2.6 v1 = a11 a12 a1n

Example 2.2.4 u1 = (1, 2, 3), u2 = (0, 1, 1), u3 = (0, 0, 6) are linearly independent since 1 2 0 0 3 6

0 1 1 = 6 = 0.

2.3

Basis and dimension

Denition 2.3.1 The vectors v1 , . . . , vn in a vector space V is called a basis of V if (i) v1 , . . . , vn are linearly independent; (ii) v1 , . . . , vn span V , i.e., V = v1 , . . . , vn .

Example 2.3.1 e1 = (1, 0, 0), e2 = (0, 1, 0), e3 = (0, 0, 1) is a basis of R3 since {e1 , e2 , e3 } is linearly independent and (a, b, c) = ae1 + be2 + ce3 a, b, c R.

{e1 , e2 , e3 } is called the standard basis of R3 . There are many other bases for R3 . For any nonzero scalar c R, {ce1 , e2 , e3 } is a basis of R3 . {u1 , u2 , u3} is a basis of R3 where (a) u1 = (1, 1, 1), u2 = (1, 1, 0), u3 = (1, 0, 1); 25

(b) u1 = (1, 2, 3), u2 = (0, 1, 1), u3 = (2, 0, 1). Let e1 = (1, 0, . . . , 0), e2 = (0, 1, 0, . . . , 0), . . . , en = (0, . . . , 0, 1) Rn . Then {e1 , e2 , . . . , en } is a basis (standard) of Rn . 1 0 0 1 0 0 0 0 0 1

0 0 0 0 1 0 be matrices in M2 (F ). Then E11 , E12 , E21 , E22 is a basis of M2 (F ). Solution: If c1 E11 + c2 E12 + c3 E21 + c4 E22 = 0, then c1 0 0 0 + 0 c2 0 0 + 0 0 c3 0 + 0 0 0 c4 = c1 c2 c3 c4 =

Example 2.3.2 Let E11 =

, E12 =

, E21 =

, E22 =

0 0 0 0

and hence c1 = c2 = c3 = c4 = 0. E11 , E12 , E21 , E22 are linearly independent. a11 a12 If A = M2 (F ), then a21 a22 A = a11 E11 + a12 E12 + a21 E21 + a22 E22 E11 , E12 , E21 , E22 . E11 , E12 , E21 , E22 = M2 (F ). E11 , E12 , E21 , E22 is a basis of M2 (F ). In general, let Eij Mm,n (F ) (1 i m, 1 j n) denote the matrix with all entries equal to zero except the ij th entry which is equal to 1. Note that
n m

(aij ) =
j =1 i=1

aij Eij

(aij ) Mm,n (F ).

Mm,n (F ) = Eij : 1 i m, 1 j n . Also

n m

aij Eij = 0 aij = 0 i, j

j =1 i=1

{Eij : i = 1, . . . , m, j = 1, . . . , n} is a basis of Mm,n (F ). Example 2.3.3 Let Pn be the vector space of all polynomials of degree n where n is a positive integer. Then {1, x, . . . , xn } is basis of Pn .

Theorem 2.3.1 A homogeneous system of m equations in n unknowns always has a nontrivial solution if m < n, that is, if the number of unknowns exceeds the number of equations. 26

Proof. Let C be the matrix in reduced row echelon form which is row equivalent to A. Then the homogeneous systems AX = 0 and CX = 0 are equivalent. If we let r be the number of nonzero rows of C, then r m. If m < n, we conclude that r < n. We are then solving r equations in n unknowns and can solve for r unknowns in terms of the remaining n r unknowns, the latter being free to take on any values we please. Thus, by letting one of these n r unknowns be nonzero, we obtain a nontrivial solution of CX = 0 and thus of AX = 0. Theorem 2.3.2 If v1 , . . . , vn is a basis of a vector space V , then (i) any collection of m vectors in V , where m > n, is linearly dependent; (ii) V = u1, . . . , us if s < n, u1 , . . . , us V . Discussion: (i) Suppose that v1 , v2 , v3 is a basis of V . Let u1 , . . . , u4 V . We show that u1 , u2 , u3, u4 must be linearly dependent. Since V = v1 , v2 , v3 , Then u1 , u2, u3 , u4 can be written as u1 = a11 v1 + a12 v2 + a13 v3 u2 = a21 v1 + a22 v2 + a23 v3 u3 = a31 v1 + a32 v2 + a33 v3 u4 = a41 v1 + a42 v2 + a43 v3 . c1 u 1 + c2 u 2 + c3 u 3 + c4 u 4 = (a11 c1 + a21 c2 + a31 c3 + a41 c4 )v1 +(a12 c1 + a22 c2 + a32 c3 + a42 c4 )v2 +(a13 c1 + a23 c2 + a33 c3 + a43 c4 )v3 . Consider the system of equations a11 c1 + a21 c2 + a31 c3 + a41 c4 = 0 a12 c1 + a22 c2 + a32 c3 + a42 c4 = 0 a13 c1 + a23 c2 + a33 c3 + a43 c4 = 0 . The system (2.6) has 4 unknowns and 3 equations. (2.6) has a nontrivial solution (d1 , d2 , d3 , d4 ) such that d1 u1 + d2 u2 + d3 u3 + d4 u4 = 0. Thus, u1 , u2 , u3, u4 is linearly dependent. (ii) Can you use the above method to show that if v1 , v2 , v3 is a basis of V , then for any two vectors u1, u2 V , u1 , u2 = V ? 27 (2.6)

Theorem 2.3.3 If u1 , . . . , uk and v1 , . . . , vm are two bases of a vector space V , then k = m. Proof. Since u1 , . . . , uk is a basis of V and v1 , . . . , vm are linearly independent, it follows from Theorem 2.3.2 (i) that m k . Since v1 , . . . , vm is a basis of V and u1 , . . . , uk are linearly independent, it follows from Theorem 2.3.2 (i) that k m. Hence k = m. Denition 2.3.2 A vector space V is called nite dimensional if there exists a nite number of vectors v1 , . . . , vk V such that V = v1 , . . . , vk . A vector space V is called innite dimensional if it is not nite dimensional, i.e., there does not exist a nite set S which spans V .

Example 2.3.4 Rn , Mm,n (R), Pn are all nite dimensional vector spaces over R. F (, ), R[x] are innite dimensional.

Remark 2.3.1 Suppose that V = v1 , . . . , vk , V = {0}. Let {vji , . . . , vjs } be a maximal subset of {v1 , . . . , vk } which is linearly independent. Then for any vector vi / {vji , . . . , vjs }, we nd that vji , . . . , vjs , vi are linearly dependent and hence vi vji , . . . , vjs . This implies that vji , . . . , vjs = V and hence vji , . . . , vjs is a basis of V . Denition 2.3.3 Let V be a vector space. If V has a basis consisting of n vectors, then we say that V has dimension n and we write dim V = n. The vector space consisting of only one zero vector is said to have dimension 0.

Example 2.3.5 (i) dim Rn = n since e1 = (1, 0, . . . , 0), e2 = (0, 1, 0, . . . , 0), . . . , en = (0, . . . , 0, 1) is a basis of Rn . (ii) dim Mm,n (F ) = m n since {Eij : i = 1, . . . , m, j = 1, . . . , n} is a basis of Mm,n (F ). (iii) dim Pn = n + 1 since 1, x, . . . , xn is a basis of Pn . 28

Example 2.3.6 Find dim Vi if a d e (i) V1 = d b f : a, b, c, d, e, f R ; e f c 0 a b (ii) V2 = a 0 c : a, b, c R . b c 0 (i) 0 0 0 0 0 , 0 1 0 0 0 0 0 0 0 0

Solution: 1 0 0 0 0 0

is a basis of V1 . (Why?) dim V1 = 6.

, 0 0 0 , 1 0 0 , 0 0 0 , 0 0 1 0 0 1 0 0 0 1 0 0 0 1 0

0 1 0

0 0 1

0 0 0

(ii)

is a basis of V2 . (Why?) dim V2 = 3.

0 0 0 0 0 1 0 1 0 1 0 0 , 0 0 0 , 0 0 1 0 1 0 1 0 0 0 0 0

(V1 V2 = {0}, M3 (R) = V1 V2 , dim M3 (R) = 9, dim M3 (R) = dim V1 + dim V2 ). Question: What is the dimension of the vector space {A M3 (R) : tr A = 0}? Example 2.3.7 Determine N (A), the null space of A and its dimension if A= 1 1 1 0 2 1 0 1 0 0 1 1 1 0 2 1 0 1 M2,4 (R). 1 0 1 0 1 2 1 1 0 0

Solution:

reduced row echelon

x1 x3 + x4 = 0 x1 =

x3 x4

x2 + 2x3 x4 = 0 x2 =2x3 + x4 1 s t 2 2s + t N (A) = , : s, t R = s 1 0 t 29

1 . 0 1

1 1 2 1 Hence, , is a basis of N (A) since these two vectors are linearly inde 1 0 0 1 pendent. dim N (A) = 2. (Note that the number of column of A = the no. of nonzero rows in its echelon form + dim N (A).) Lemma 2.3.1 (Lemma A) Let v1 , . . . , vk be k linearly independent vectors in a vector space V and v V . Then v1 , . . . , vk , v are linearly independent (Exercise) Theorem 2.3.4 Let V be a vector space of dim n and v1 , . . . , vn are n vectors in V . If (i) v1 , . . . , vn are linearly independent or (ii) V = v1 , . . . , vn , then v1 , . . . , vn is a basis of V . Proof. (i) Suppose that v1 , . . . , vn are linearly independent. We wish to show that V = v1 , . . . , vn . Let v V . If v / v1 , . . . , vn , then by Lemma A, v1 , . . . , vn , v are linearly independent. This is not possible since V has a basis consisting n vectors (Theorem 2.3.2). (ii) Suppose that v1 , . . . , vn span V . We wish to show that v1 , . . . , vn are linearly independent. Suppose that v1 , . . . , vn are linearly dependent. By Theorem 2.2.1, one of the vi s, say vn , can be written as a linear combination of v1 , . . . , vn1 . It follows that v1 , . . . , vn1 = v1 , . . . , vn = V. This contradicts Theorem 2.3.2 (ii). Remark 2.3.2 In the proof of (ii), we use the fact: if w is a linear combination of v1 , . . . , vk , then v1 , . . . , vk , w = v1 , . . . , vk . v / v1 , . . . , vk .

30

Remark 2.3.3 Theorem 2.3.2 (ii) can also be shown by using Theorem 2.3.3. Theorem 2.3.2 (ii) can now be stated as follows: If dim V = n, then no set of less than n vectors can span V . Solution: (Method of Contradiction) Suppose there are k vectors v1 , . . . , vk that span V where k < n. If v1 , . . . , vk are linearly independent, then v1 , . . . , vk will be a basis, contradicting dim V = n. Hence v1 , . . . , vk are linearly dependent. Then one of vi s, say vk is a linear combination of v1 , . . . , vk1, i.e., vk v1 , . . . , vk1 . Hence v1 , . . . , vk1 = v1 , . . . , vk1 , vk = V . If v1 , . . . , vk1 are linearly independent, then v1 , . . . , vk1 is a basis, a contradiction to the fact that dim V = n. If v1 , . . . , vk1 are linearly dependent, we can eliminate another vector and still have a spanning set. We can continue eliminating vectors in this way until we arrive at a linearly independent spanning set with < n elements. But this contradicts to the fact that dim V = n. Hence no set of less than n elements can span V . Theorem 2.3.5 Let V be a vector space of dimension n. Let v1 , . . . , vk be k linearly independent vectors in V where k < n. Then v1 , . . . , vk can be extended to a basis v1 , . . . , vk , . . . , vn of V . Proof. Since k < n, v1 , . . . , vk cannot be a basis of V . Therefore v1 , . . . , vk = V . Let vk+1 V \ v1 , . . . , vk . By Lemma A, v1 , . . . , vk , vk+1 are linearly independent. If k + 1 < n, then in a similar manner, v1 , . . . , vk , vk+1 can be extended to k + 2 linearly independent vectors. This process may be continued until a set {v1 , . . . , vk , vk+1 , . . . , vn } of n linearly independent vectors is obtained. 1 2 1 Example 2.3.8 Show that 2 , 1 , 0 is a basis of M3,1 (R). 3 0 1

Solution: Since dim M3,1 (R) = 3, we need only to show that these 3 vectors are linearly independent. That follows since 1 2 1 2 3 1 0 0 = 2 = 0. 1

Example 2.3.9 Extend u1 = (1, 4, 2), u2 = (3, 1, 6) to a basis of R3 . 31

Solution: Take u3 = (0, 0, 1). Since 4 1 0 = 13 = 0, therefore u1 , u2 , u3 are linearly 2 6 1 independent. By Theorem 2.3.4, u1 , u2, u3 is a basis of R3 . Example 2.3.10 Extend u1 = (1, 2, 3, 4), u2 = (5, 6, 7, 8) to a basis of R4 . Solution: 1 2 3 4 7 8 2 3 4
2 1

R 5R

4 rows of B are linearly independent 4 rows of A are linearly independent. Hence, (1, 2, 3, 4), (5, 6, 7, 8), (0, 0, 1, 0), (0, 0, 0, 1) is a basis of R4 .

5 6 1 5 A= 0 0

1 6 7 8 0 0 1 0 0 0 0 1 0

. 0 4 8 12 2 3 4 4 8 12 = B (row echelon form) 0 1 0 0 0 1

2.4
Then

Dimension of subspaces

Theorem 2.4.1 Let V be a subspace of a vector space W and W is nite dimensional.

(i) dim V dim W ; (ii) dim V = dim W V = W.

Proof. (i) Let v1 , . . . , vn be a basis of V . Then v1 , . . . , vn are linearly independent. By Theorem 2.3.2, n dim W . dim V dim W . (ii) (Exercise) Remark 2.4.1 If V M2,3 (R), then dim V 6. If V R4 , then dim V 4. Theorem 2.4.2 Let W be a nite dimensional vector space and U, V W such that W = U V . Then dim W = dim U + dim V . Hints of Proof. Let u1 , u2, . . . , uk be a basis of U and v1 , v2 , . . . , vl be a basis of V . Then show that is a basis of W .

32

Theorem 2.4.3 Let U, V W where dim W is nite. Then dim(U + V ) = dim U + dim V dim(U V ). Hints of Proof. If U V = {0}, then this follows from Theorem 2.4.2. Suppose U V = {0}. Let x1 , . . . , xs be a basis of U V . Extend x1 , . . . , xs to a basis x1 , . . . , xs , y1 , . . . , ym of U . Extend x1 , . . . , xs to a basis x1 , . . . , xs , z1 , . . . , zn of V . Method 1 (Without using Theorem 2.4.2) Show that x1 , . . . , xs , y1, . . . , ym , z1 , . . . , zn are linearly independent and U + V = x1 , . . . , xs , y1 , . . . , ym, z1 , . . . , zn and hence dim(U + V ) = s + m + n = (s + m) + (s + n) s = dim U + dim V dim(U + V ). Method 2 (Use Theorem 2.4.2) Show that U + V = U z1 , . . . , zn . Then dim(U + V ) = dim U + dim z1 , . . . , zn =m+s+n = dim U + dim V dim(U V ). Example 2.4.1 Let U = {(x, y, x) : x, y R} and V = {(x, z, z ) : x, z R}. Find dim U , dim V and dim(U + V ). Solution: {(1, 0, 1), (0, 1, 0)} is a basis of U . dim U = 2. {(1, 0, 0), (0, 1, 1)} is a basis of V . dim V = 2. Let (Theorem 2.4.2)

(a, b, c) U V (a, b, c) U and (a, b, c) V a = c, b = c a = b = c. U V = {(a, a, a) : a R} = (1, 1, 1) . dim(U V ) = 1. dim(U + V ) = dim U + dim V dim(U V ) = 2 + 2 1 = 3. (Or observe that (0, 1, 0) U U + V, (1, 0, 0) V U + V , (0, 1, 1) + (1)(0, 1, 0) = (0, 0, 1) U + V. 33

Hence U + V has 3 linearly independent vectors (0, 1, 0), (1, 0, 0) and (0, 0, 1). 3 dim(U + V ) dim R3 = 3 (Note that U + V = R3 by Theorem 2.4.1 (ii).) a11 ... a1n dim(U + V ) = 3.

a . . . a 2n 21 Denition 2.4.1 Let A = (aij ) = . Mm,n (F ) and . .. . . . . . am1 . . . amn

u1 = (a11 , . . . , a1n ), u2 = (a21 , . . . , a2n ), . . . , um = (am1 , . . . , amn ).

Then u1, . . . , um is called the row space of A.

. Then the row space of A is (1, 0, 0), (0, 1, 0), (0, 0, 0) . Example 2.4.2 Let A = 0 1 0 0 0 0 It is equal to {(x, y, 0) : x, y R}, the xy -plane.

1 0 0

Remark 2.4.2 (i) u1, u2 , u3 = u1, u3 , u2 ; (ii) u1, u2 , u3 = u1, u2 + u1 , u3 ; (iii) u1, u2 , u3 = u1, u2 , u3 if = 0. Theorem 2.4.4 If A B , A, B Mm,n (F ), then row space of A = row space of B .

Theorem 2.4.5 Let E1 and E2 be reduced row echelon matrices where E1 Mm,p (F ), E2 Mn,p (F ). Then row space of E1 = row space of E2 E1 , E2 have the same nonzero rows.

34

 Example 2.4.3 E1 = 0 0 0 1 , E2 = 0 0 1 0 are reduced row echelon 0 0 0 0 0 0 0 0 matrices. Row space of E1 = row space of E2 (Why?).

0 1 2 0

0 1 0 0

Corollary 2.4.1 Let A Mm,p (F ), B Mn,p (F ). Then the row space of A = row space of B the reduced row echelon matrix of A and the reduced row echelon matrix of B have the same non-zero rows.

Example 2.4.4 Let U = (1, 2, 1, 3), (2, 4, 1, 2), (0, 0, 6, 16) and W = (1, 2, 4, 11), (2, 4, 5, 14) . Is W = U ? Solution: U 1 A= 2 0
1 R 3 2 0 0

is the row space of A where 2 1 3 1 2 1 3 R2 2R1 4 1 2 0 0 3 8 0 6 16 0 0 6 16 3 1 0 1 2 0

1 3

1 2 1

0 0

W is the row space of B where B= 1 2 4 11 2 4 5 14

R1 +4R2

R1 +R2 0 0 1 8 = E1 8 3 3 0 0 0 0 0
R 2R

R3 2R2 0 0 0 0

1 2 1 3 0

8 0

(reduced row echelon matrix)

2 1

1 2 4 11 0 0 3 8

R2

1 2 4 0 0 1

11
8 3

1 2 0

1 3

0 0 1 8 3

= E2

(reduced row echelon matrix)

The nonzero rows of E1 and E2 are identical. row space of A = row space of B . Thus, U = W . Remark 2.4.3 If U = u1 , . . . , uk , W = w1 , . . . , ws , then U + W = u 1 , . . . , u k , w1 , . . . , ws .

35

Example 2.4.5 Let V = (1, 1, 1, 1), (0, 0, 1, 1) , W = (1, 0, 0, 0), (2, 2, 0, 0) . Find a basis for U + W and extend it to a basis of R4 . What is the dimension of U W ? Solution: U + W = 1 1 1 0 0 1 A= 1 0 0 2 2 0 (1, 1, 1, 1), (0, 0, 1, 1), (1, 0, 0, 0), (2, 2, 0, 0) = row space of 1 1 1 1 1 1 1 1 1 1 1 R3 R1 0 0 R2 R3 0 1 1 R4 2R1 0 1 1 1 0 0 0 1 0 0 0 2 2 0 0 2 1 1 1 1 A where 1 1 1 2

U + W = (1, 1, 1, 1), (0, 1, 1, 1), (0, 0, 1, 1) and dim(U + W ) = 3 since {(1, 1, 1, 1), (0, 1, 1, 1), (0, 0, 1, 1)} is linearly independent. 1 1 1 1

0 1 1 1 0 0 1 1 0 0 0 0
R4 +2R3

(row echelon matrix)

(0, 1, 1, 1), (0, 0, 1, 1), (0, 0, 0, 1)} is a basis of R4 . This gives us that dim(U + W ) = dim U + dim W dim(U W ) 3 = 2 + 2 dim(U W ). Hence dim(U W ) = 1. a11 a12 a21 a22 Denition 2.4.2 Let A = . . . . . . am1 am2 ... a1n

0 1 1 1 Note that is a row echelon matrix and hence {(1, 1, 1, 1), 0 0 1 1 0 0 0 1

Then

a11 a12 a1n a a a 21 22 2n . , . ,..., . . . . . . . am1 am2 amn 36

a2n Mm,n (F ). . . . . . . amn ... .. .

is called the column space of A.

 Example 2.4.6 Let A = 2 5 2 1 . Then the column space W of A is 3 6 3 1 2 , 5 , 2 , 1 . 3 6 3 1 1 and dim W = 2. 1 4 1 1

1 4 1 1

Note that W =

2 , 1 3 1 1

Row space of A = (1, 4, 1, 1), (0, 3, 0, 1) and has dim 2. This shows that dim(row space of A) = dim(column space of A).

R2 2R1 2 5 2 1 R 0 3 3 3R1 3 6 3 1 0 6

1 4 1 1

1 0 0

R3 2R2 1 0 3 2 0 0

1 0 0

1 0

Remark 2.4.4 a11 a12 a13 x1 a11 x1 + a12 x2 + a13 x3 a21 a22 a23 x2 = a21 x1 + a22 x2 + a23 x3 a31 a32 a33 x3 a31 x1 + a32 x2 + a33 x3 a11 a12 a13 = x1 a21 + x2 a22 + x3 a23 column space of (aij ). a31 a32 a33 b1 x1 a11 a12 a13 The system of linear equation a21 a22 a23 x2 = b2 has a solution b3 x3 a31 a32 a33 b1 b2 b3 is .

37

 a1j . . In general, if A = (aij ) Mm,n (F ), Cj = . is the j th column of A. Then amj x 2 A . = x1 C1 + x2 C2 + + xn Cn . . . xn Theorem 2.4.6 A system of linear equations Ax = b is consistent b column space of A. x1

Theorem 2.4.7 If A Mm,n (F ), then dim(row space of A) = dim(column space of A).

Remark 2.4.5 If W = w1 , . . . , ws = {0}, then the maximum number of linearly independent vectors in {w1 , . . . , ws } is the dimension of W .

Denition 2.4.3 If A Mm,n (F ), then dim(row space of A) is called the rank of A and is denoted by (A).

Remark 2.4.6 If A E and E is an echelon matrix, then

row

(A) = the number of nonzero rows in E .

Remark 2.4.7 (A) is the maximum number of linearly independent vectors in A. 1 0 Example 2.4.7 A = 1 0 0 1 1 0 0 0 1 0

1 0 0 1

0 0

1 0 = E , (A) = (E ) = 3. 0 1 0 0 38

2.5

Ordered basis and coordinates

Theorem 2.5.1 Let {u1 , . . . , un } be a basis of a vector space U over a eld F . Then for any u U , there exist unique scalars c1 , c2 , . . . , cn such that u = c1 u1 + c2 u2 + cn un . Proof. Since u U and U = u1 , . . . , un , it follows that u = ci F, i = 1, . . . , n. Suppose that u =
n i=1 ci ui n i=1 ci ui

for some

=
n

n i=1

di ui for some ci , di F, i = 1, . . . , n. Then

n

ci u i
i=1 n i=1

di u i = 0

(c i d i )u i = 0 .
i=1

Since u1 , . . . , un are linearly independent, it follows that ci di = 0 i = 1, . . . , n ci = di, i = 1, . . . , n. (Expression of u as a linear combination of u1 , . . . , un is unique.) Up to now the order of the vectors in a basis has been of no importance. However, in this section, we shall speak of an ordered basis {u1 , . . . , un } which just a basis where the order in which the vectors are listed is taken into account. Note that the ordered basis {u1 , u2, u3 , u4 } of R4 is not the same as the ordered basis {u 2 , u 1 , u 3 , u 4 }. Denition 2.5.1 Let S = {u1 , . . . , un } be an ordered basis of a vector space U . If u = c1 u1 + + cn un , then (i) c1 , c2 , . . . , cn are called the coordinates of u with respect to the ordered basis S ; c1 c 2 (ii) [u]s = . is called the coordinate vector of u with respect to the ordered basis S . . . cn Example 2.5.1 Let S = {(1, 1), (2, 1)}, T = {(1, 0), (0, 1)} be two ordered basis of R2 . Let w = (8, 6). Then w = 8(1, 0) + 6(0, 1) w = 4(1, 1) + 2(2, 1). 39

We have [w ]T =

8 6

, [w ]S =

4 2

u1 = (1, 1) = 1e1 + 1e2 u2 = (2, 1) = 2e1 + 1e2 and 8 6 1 2 1 1 = 1 2 1 1 4 2 .

where e1 = (1, 0), e2 = (0, 1).

is called the transition matrix from the ordered basis S to the ordered basis T .

Denition 2.5.2 Let S = {u1 , . . . , un } and T = {v1 , . . . , vn } be two ordered bases of a vector space U . If u1 = a11 v1 + + a1n vn u2 = a21 v1 + + a2n vn . . ., un = an1 v1 + + ann vn then the matrix (aij )t is called the transition matrix from the basis S to the basis T and is denoted by [P ]T S.

Example 2.5.2 Let S = {u1 = (3, 7), u2 = (1, 1)} T = {v1 = (2, 3), v2 = (1, 4)} be two ordered bases of R2 and w = (9, 11). Find (i) the transition matrix from basis S to the basis T ; (ii) the coordinate vector of w relative to the basis S ; (iii) the coordinate vector of w relative to the basis T . 40

Solution: (i) (3, 7) = 1(2, 3) + 1(1, 4) (1, 1) = 1(2, 3) + (1)(1, 4) (ii) w = (9, 11) = 2(3, 7) + 3(1, 1) [w ]S = 2 3 5 1 . [P ]T S = 1 1
t

1 1

1 1

(iii) 5 1

w = (9, 11) = 5(2, 3) + (1)(1, 4) 1 1 2 3

[w ]T =

Note that

1 1

, i.e., [w ]T = [P ]T S [w ]S .

Theorem 2.5.2 Let S and T be two ordered bases of a nite dimensional vector space V . Then (i) [P ]T S [w ]S = [w ]T where w V .
S (ii) [P ]T S [P ]T = identity matrix.

Proof of (ii). For any v V , we have [P ]T S [v ]S = [v ]T [P ]S T [v ]T = [v ]S From (2.7) and (2.8),

S T [P ]T S [P ]T [v ]T = [P ]S [v ]S = [v ]T S Since {[v ]T : v V } = Mn,1 (F ), it follows that [P ]T S [P ]T = I .

(2.7) (2.8)

(2.9)

Remark 2.5.1 We have used the following fact in the proof of Theorem 2.5.2 (ii). If A Mn (F ) such that Ay = y y Mn,1 (F ), then A = In .

Example 2.5.3 Let S = {u1 = (6, 3, 3), u2 = (4, 1, 3), u3 = (5, 5, 2)} T = {v1 = (2, 0, 1), v2 = (1, 2, 0), v3 = (1, 1, 1)} be two ordered bases of R3 . Find 41

(i) the transition matrix [P ]T S from the basis S to the basis T ; (ii) the coordinate vector [w ]S where w = (4, 9, 5); (iii) the coordinate vector [w ]T where w = (4, 9, 5). Solution: Let (x, y, z ) = 1 v1 + 2 v2 + 3 v3 = 1 (2, 0, 1) + 2 (1, 2, 0) + 3 (1, 1, 1). Then 1 1 = (2x y z ) 3 1 2 2 + 3 = y 2 = (x + y 2 z ) 3 1 1 + + 3 = z 1 = (2x + y + 4z ). 3 1 1 1 (x, y, z ) = 3 (2x y z )v1 + 3 (x + y 2z )v2 + 3 (2x + y + 4z )v3 . 1 1 1 u1 = (6, 3, 3) = (12 3 3)v1 + (6 + 3 6)v2 + (12 + 3 + 12)v3 3 3 3 = 2v1 + v2 + v3 2 1 + 2 + 3 = x u2 = (4, 1, 3) = 2v1 v2 + v3 u3 = (5, 5, 2) = v1 + 2v2 + v3 t 2 1 1 2 2 1 2 1 1 = 1 1 2 . [P ]T = S 1 2 1 1 1 1

1 We obtain x3 = 2, x2 = 2, x1 = 1, [w ]S = 2 . 2 42

2 2 1 x1 4 1 1 2 x2 = 5 , 1 1 1 x3 1

1 1 1 w = (4, 9, 5) = (8 + 9 5)v1 + (4 9 10)v2 + (8 9 + 20)v3 3 3 3 = 4v1 5v2 + v3 4 [w ]T = 5 . 1 1 1 1 1 1 2 2 2 1 1 1 1 R2 R1 0 2 1 5 R3 2R1 4 0 0 1 1 1

6 2

Chapter 3 Linear Transformations

3.1 Linear transformation (mappings, operators)

Let U and V be vector spaces over a eld F . A mapping L : U V is called a linear transformation if (i) L(x + y ) = L(x) + L(y ) x, y U ; (ii) L(x) = L(x) F, x U . If U = V , L is called a linear transformation on U . Example 3.1.1 Let L : R2 R2 be dened by L(u) = 3u, u R2 . Then L(u + v ) = 3(u + v ) = 3u + 3v = L(u) + L(v ) u, v R2 ; L(u) = 3(u) = (3u) = L(u) R, u R2 . L is a linear mapping. y L(u) = 3u

43

Example 3.1.2 Let L : R2 R2 be dened by L((x1 , x2 )) = (x1 , 0), (x1 , x2 ) R2 . Then L is a linear mapping. Solution: Note that L ((x1 , x2 ) + (y1 , y2)) = L ((x1 + y1 , x2 + y2 )) = (x1 + y1 , 0) = (x1 , 0) + (y1 , 0) = L((x1 , x2 )) + L((y1 , y2 )); L ((x1 , x2 )) = L ((x1 , x2 )) = (x1 , 0) = (x1 , 0) = L((x1 , x2 )) (x1 , x2 ), (y1 , y2) R2 , R. Example 3.1.3 Let L : R2 R3 be dened by L((x, y )) = (x, y, x + y ). Then L is a linear mapping. Solution: For any u = (x1 , x2 ), v = (y1 , y2 ) R2 , R, note that L(u + v ) = L ((x1 , x2 ) + (y1 , y2 )) = L ((x1 + y1 , x2 + y2 )) = (x1 + y1 , x2 + y2 , (x1 + y1 ) + (x2 + y2 )) = (x1 , x2 , x1 + x2 ) + (y1 , y2, y1 + y2 ) = L(u) + L(v ); L(u) = L ((x1 , x2 )) = L ((x1 , x2 )) = (x1 , x2 , x1 + x2 ) = (x1 , x2 , x1 + x2 ) = L(u). Example 3.1.4 Let L : R2 R2 be dened by L((x, y )) = (x + 1, y ). Then L ((1, 0)) = (1 + 1, 0) = (2, 0) L (2(1, 0)) = L((2, 0)) = (2 + 1, 0) = (3, 0) 2L ((1, 0)) = 2(2, 0) = (4, 0) = L (2(1, 0)) . L is not a linear map. Theorem 3.1.1 Let U and V be vector spaces over a eld F . A mapping L : U V is linear L(au + bw ) = aL(u) + bL(w ) a, b F, u, w U. Remark 3.1.1 Let L : U V be a linear map and u1 , . . . , uk U , 1 , . . . , k F . Then L(1 u1 + + k uk ) = 1 L(u1 ) + + k L(uk ). 44

3.2

Properties of linear mappings

Let L : U V be a linear mapping. Then (i) L(0) = 0; (ii) L(u1 u2 ) = L(u1 ) L(u2 ); (iii) L(u) = L(u).

Proof. (i) L(0) = L(0u) = 0L(u) = 0 u U, 0 F . (ii) L(u1 u2 ) = L(u1 + (1)u2 ) = L(u1 ) + L((1)u2 ) = L(u1 ) + (1)L(u2 ) = L(u1 ) L(u2 ). (iii) L(u) = L(0 u) = L(0) L(u) = 0 L(u) = L(u).

3.3

Linear mappings from Mn,1(F ) to Mm,1 (F )

Let A Mm,n (F ). Dene a mapping LA : Mn,1 (F ) Mm,1 (F ) by LA (x) = Ax, x Mn,1 (F ). Then LA (ax + by ) = A(ax + by ), a, b F, x, y Mn,1 (F )

= A(ax) + A(by ) = aAx + bAy = aLA (x) + bLA (y ). LA is a linear map. Example 3.3.1 Let A = M2,1 (R) by LA x y = cos sin sin cos x y = x cos y sin x sin + y cos . cos sin sin cos , R is xed. Dened LA : M2,1 (R)

Then LA is a linear map and is called a rotation map on R2 . y LA x y x y

0 45

46

Theorem 3.3.1 Let {u1 , . . . , un } be a basis of a vector space U over a eld F . Let v1 , . . . , vn be n vectors in a vector space V over F . Then there exists a unique linear mapping L : U V such that (ui ) = vi , i = 1, . . . , n. Proof. (a) For any u U , u = L
i=1 n i=1 ci ui . n

Let L : U V be a mapping such that

n

ci u i 47

=
i=1

ci vi .

(3.1)

Then we have L(ui ) = vi , i = 1, . . . , n. (i) For any x, y U , we have x =

n n i=1 ci ui n

and y =

n i=1

di ui . Hence,
n

L(x + y ) = L
i=1 n

ci u i +
i=1

di u i

=L
i=1

(c i + d i )u i

=
i=1 n

(ci + di)vi (ci vi + di vi ) =

i=1 n

(by denition of L: (3.1))

n n

= =L

ci vi +
i=1 n i=1

di vi

ci u i
i=1

+L
i=1

di u i

= L(x) + L(y ). (ii) For any F ,

n n

L(x) = L
i=1 n

ci u i (ci )ui

=L
i=1

(c i u i )

=L
i=1 n

=
i=1

(ci )vi =
i=1 n

(ci vi ) =
i=1

ci vi

= L
i=1

ci u i

= L(x).

L is a linear map. (b)Uniqueness: Suppose that M : U V is a linear map such that M (ui ) = vi , i = 1, . . . , n. Then
n n n

M
i=1

ci u i

=
i=1

c i M (u i ) =
i=1

ci vi

ci F, i = 1, . . . , n.

L = M.

Theorem 3.3.2 Every linear mapping from Mn,1 (F ) Mm,1 (F ) is of the form LA for some matrix A Mm,n (F ). 48

 0 1 0 1 . . 0 . , . . . , en = Proof. Let e1 = . , e2 = 0 . Then . . 0 . . . 0 1 0 a2j (ej ) = . . . anj a1j

for some aij , i = 1, . . . , m, j = 1, . . . , n.

Take A = (aij ) Mm,n (F ). Then 0 . . . a1n 0 a2n 1 j th entry . . . 0 amn . . . 0 j = 1, . . . , n.

a11 . . . a 21 . . . LA (ej ) = Aej = . .. . . . am1 . . . a1j . . = . , anj LA .

Since (ej ) = LA (ej ) and e1 , . . . , en is a basis, it follows from Theorem 3.3.1 that =

Example 3.3.2 Let : M3,1 (R) M3,1 (R) be dened by x 3x + y y = 2x + z . z 4x + z Then 1 3 0 1 0 0 0 = 2 , 1 = 0 , 0 = 1 . 0 4 0 0 1 1 49

 x 3 1 0 x 3 1 0 y = 2 0 1 y = LA , where A = 2 0 1. z 4 0 1 z 4 0 1

3.4

Range and null space of a linear map

Let L : U V be a linear map. Let R(L) = {L(u) : u U }. Then R(L) is called the range of L. (It is also called the image of L and is sometimes written as Im L.) Let N (L) = {u U : L(u) = 0}. Then N(L) is the set of all vectors from U whose images are zero vector. N (L) is called the null space of L. It is also called the kernel of L. U V R(L)

U N (L)

Theorem 3.4.1 Let L : U V be a linear map. Then (i) N (L) U and (ii) R(L) V . Proof. (i) L(0) = 0, 0 N (L). Let x, y N (L), F . Then L(x) = L(y ) = 0. L(x + y ) = L(x) + L(y ) = 0 + 0 = 0 x + y N (L). L(x) = L(x) = 0 = 0 x N (L). N (L) U . 50

(ii) L(0) = 0, 0 R(L). Let v, w R(L), F . Then there exist some x, y U such that v = L(x), w = L(y ). L(x + y ) = L(x) + L(y ) = v + w v + w R(L). L(x) = L(x) = v v R(L). Hence R(L) V .

Example 3.4.1 Let L : R3 R3 be dened by L ((x, y, z )) = (x, y, 0). Then R(L) = {(x, y, 0) : x, y R} = xy plane N (L) = {(x, y, z ) R3 : L ((x, y, z )) = 0} = {(x, y, z ) R3 : (x, y, 0) = (0, 0, 0)} = {(x, y, z ) R3 : x = y = 0} = z axis. Note that dim R(L) = 2, dim N (L) = 3.

Denition 3.4.1 Let L : U V be a linear map. Then dim R(L) is called the rank of L and dim N(L) is called the nullity of L.

Theorem 3.4.2 If L : U V is a linear map and {u1 , . . . , un } is a basis of U , then R(L) = L(u1 ), . . . , L(un ) . Proof. Let v L(u1 ), . . . , L(un ) .
n

v=
i=1 n

ci L(ui )

for some c1 , . . . , cn F
n

=
i=1

L(ci ui ) = L
i=1

ci u i

v R(L) L(u1 ), . . . , L(un ) R(L). 51

We shall show that R(L) L(u1 ), . . . , L(un ) . Let v R(L). Then v = L(u) for some u U .
n

=L
i=1

ci u i

(since U = u1 , . . . , un ) for some c1 , . . . , cn F

=
i=1 n

L(ci ui ) ci L(ui ) L(u1 ), . . . , L(un ) .

i=1

R(L) L(u1 ), . . . , L(un ) . Thus, R(L) = L(u1 ), . . . , L(un ) . Example 3.4.2 Find the linear mapping L : R2 R2 such that L((1, 1)) = (1, 4), L((0, 1)) = (3, 4). Solution: Consider (x, y ) = c1 (1, 1) + c2 (0, 1). Then (x, y ) = (c1 , c1 + c2 ). This gives us that c1 = x, c2 = y c1 = y x. (x, y ) = x(1, 1) + (y x)(0, 1).

L((x, y )) = L (x(1, 1) + (y x)(0, 1)) = xL((1, 1)) + (y x)L((0, 1)) = x(1, 4) + (y x)(3, 4) = (x + 3y 3x, 4x + 4y 4x) = (2x + 3y, 4y ). Example 3.4.3 Find a linear transformation L : R3 R4 such that R(L) is spanned by (1, 2, 0, 4), (2, 0, 1, 3). Solution: Let L : R3 R4 be the linear map such that L((1, 0, 0)) = (1, 2, 0, 4) L((0, 1, 0)) = (2, 0, 1, 3). L((0, 0, 1)) = (0, 0, 0, 0) 52

Then R(L) = (1, 2, 0, 4), (2, 0, 1, 3) . L((x, y, z )) = L (x(1, 0, 0) + y (0, 1, 0) + z (0, 0, 1)) = xL((1, 0, 0)) + yL((0, 1, 0)) + zL((0, 0, 1)) = x(1, 2, 0, 4) + y (2, 0, 1, 3) + z (0, 0, 0, 0) = (x + 2y, 2x, y, 4x 3y ). Theorem 3.4.3 Let L : U V be a linear mapping where U is nite dimensional. Then (L) + (L) = dim U where (L) = rank of L, (L) = nullity of L. Proof. Note that (L) = dim R(L), (L) = dim N (L). Let dim U = n, (L) = k . Let u1 , . . . , uk be a basis of N (L). Then L(ui ) = 0 i = 1, . . . , k . Extend u1 , . . . , uk to a basis u1, . . . , uk , uk+1, . . . , un to U . Note that R(L) = L(u1 ), . . . , L(uk ), L(uk+1), . . . , L(un ) = 0, . . . , 0, L(uk+1), . . . , L(un ) = L(uk+1), . . . , L(un ) . We shall show that L(uk+1 ), . . . , L(un ) are linearly independent. Suppose that ck+1 L(uk+1) + ck+2 L(uk+2 ) + + cn L(un ) = 0, ci F, i = 1, . . . , n. Then L (ck+1 uk+1) + L (ck+2 uk+2) + + L (cn un ) = 0 ci F, i = 1, . . . , n. L (ck+1 uk+1 + + cn un ) = 0 ck+1 uk+1 + + cn un N (L) = u1 , . . . , uk ck+1 uk+1 + + cn un = c1 u1 + + ck uk for some c1 , . . . , ck F c1 u1 + + ck uk + (ck+1 )uk+1 + + (cn )un = 0 c1 = = ck = ck+1 = = cn = 0 ( u1 , . . . , un is a basis of U ) L(uk+1), . . . , L(un ) are linearly independent. L(uk+1 ), . . . , L(un ) is a basis of R(L). dim R(L) = (L) = n k n = dim U = (L) + k = (L) + (L). 53

Example 3.4.4 Let L : R4 R3 be the linear mapping dened by L((x, y, s, t)) = (x y + s + t, x + 2s t, x + y + 3s 3t). Find (i) a basis of R(L); (ii) a basis of N (L); (iii) rank of L; (iv) nullity of L.

Solution: (i) & (iii) u1 = (1, 0, 0, 0), u2 = (0, 1, 0, 0), u3 = (0, 0, 1, 0), u4 = (0, 0, 0, 1) is a basis of R4 . Hence L(u1 ) = (1, 1, 1), L(u2 ) = (1, 0, 1), L(u3 ) = (1, 2, 3), L(u4 ) = (1, 1, 3). In view of Theorem 3.4.2, R(L) = L(u1 ), L(u2 ), L(u3), L(u4 ) = (1, 1, 1), (1, 0, 1), (1, 2, 3), (1, 1, 3) = row space of A where 1 1 1 1 1 1 1 1 0 2 0 1 R2 +R1 0 1 A= 0 R3 R1 0 1 1 2 2 3 R4 R1 0 0 2 4 1 1 3 (ii) & (iv) xy+s+t= 0 x + 2s t = 0 x + y + 3s 3 t = 0 1 1 1 1 0 1 0 2 1 0 0 2 1 0 0 1 1 2 1 1 3 3 0 0 0 0 0 x + 2s t = 0 y + s 2t = 0

1 1

{(1, 1, 1), (0, 1, 2)} is a basis for R(L). Hence, the rank of L = 2.

1 2 0 0 0 0

(r.e.f)

0 0

x = 2s + t

y = s + 2t

N (L) = {(2s + t, s + 2t, s, t) : s, t R} = {s(2, 1, 1, 0) + t(1, 2, 0, 1) : s, t R} = (2, 1, 1, 0), (1, 2, 0, 1) {(2, 1, 1, 0), (1, 2, 0, 1)} is a basis of N (L). nullity of L = 2. (Note that dim R4 = 2 + 2 = (L) + (L).) 54

Theorem 3.4.4 Let A Mm,n (F ). Then n = (A) + (A) where (A) = dim N (A) which is called the nullity of A. Proof. Let A : Mn,1 (F ) Mm,1 (F ) be the linear map dened by A (x) = Ax, x Mn,1 (F ). Then R ( A ) = a1n a11 a12 . . , . ,..., . . . . . . . am1 am2 amn = dim row space of A = (A). From Theorem 3.4.3, dim Mn,1 (F ) = n = (A ) + (A ) = (A) + (A) since the null space of A = the null space of A. 1 0 2 1 M3,4 . Then A 0 1 1 2 Example 3.4.5 Let A = 0 0 2 1 1 1 3 3 0 0 0 0 Hence, (A) = 2, 4 = (A) + (A) = 2 + (A) (A) = 2. 1 1 1 1

rank A = dim column space of A

(r.e.f ).

Theorem 3.4.5 Let U be a vector space with a basis {u1 , . . . , un }. Let L : U V be a linear transformation. If L(u1 ) = = L(uk ) = 0 and L(uk+1 ), . . . , L(un ) are linearly independent, then {u1 , . . . , uk } is a basis of N (L) and L(uk+1 ), . . . , L(un ) is a basis of R(L). Proof. (i) R(L) = L(u1 ), . . . , L(uk ), L(uk+1), . . . , L(un ) = 0, . . . , 0, L(uk+1), . . . , L(un ) = L(uk+1), . . . , L(un ) . Since L(uk+1 ), . . . , L(un ) are linearly independent, it follows that L(uk+1 ), . . . , L(un ) is a basis of R(L). (ii) dim V = n = (L) + (L) = (n k ) + (L) (L) = k. 55 ( L(u1 ) = = L(uk ) = 0)

Since u1 , . . . , uk are linearly independent and ui N (L) for i = 1, . . . , k and dim N (L) = k , it follows that u1 , . . . , uk is a basis of N (L). Example 3.4.6 Find a linear transformation L : R3 R3 such that the null space of L = (1, 2, 1), (1, 0, 0) . Solution: Note that (1, 2, 1), (1, 0, 0), (0, 1, 0) is a basis of R3 (Why?). There exists a linear transformation L : R3 R3 such that L((1, 2, 1)) = (0, 0, 0), L((1, 0, 0)) = (0, 0, 0), L((0, 1, 0)) = (0, 1, 0). From Theorem 3.4.5, N (L) = (1, 2, 1), (1, 0, 0) . (x, y, z ) = c1 (1, 2, 1) + c2 (1, 0, 0) + c3 (0, 1, 0) = (c1 + c2 , 2c1 + c3 , c1 ) c1 = z, 2c1 + c3 = y, c1 + c2 = x c3 = y 2c1 = y 2z, c2 = x c1 = x z . (x, y, z ) = z (1, 2, 1) + (x z )(1, 0, 0) + (y 2z )(0, 1, 0). Hence L((x, y, z )) = (y 2z )(0, 1, 0) = (0, y 2z, 0). Example 3.4.7 Find a linear map L : R3 R3 such that NL = (2, 8, 1 ). 2 Solution: Note that u1 = (2, 8, 1 ), u2 = (0, 1, 0), u3 = (0, 0, 1) is a basis of R3 . Let 2 L : R3 R3 be a linear mapping such that L(u1 ) = 0, L(u2 ) = u2 , L(u3 ) = u3 . Then N (L) = (2, 8, 1 ) (Theorem 3.4.5). 2 1 (x, y, z ) = c1 (2, 8, ) + c2 (0, 1, 0) + c3 (0, 0, 1) 2 1 = (2c1 , 8c1 + c2 , c1 + c3 ) 2 1 x = 2 c1 , y = 8 c1 + c2 , z = c1 + c3 . 2 x 1 1 c1 = , c2 = y 8c1 = y 4x, c3 = z c1 = z x. 2 2 4 L((x, y, z )) = c1 L(u1 ) + c2 L(u2 ) + c3 L(u3 ) 1 = 0 + (y 4x)(0, 1, 0) + (z x)(0, 0, 1) 4 1 = (0, y 4x, z x). 4 56

Denition 3.4.2 A mapping : A B is called injective (one-to-one) if x, y A, x = y f (x) = f (y ).

Theorem 3.4.6 Let L : U V be a linear transformation where dim U < . Then the following conditions are equivalent: (i) L is injective; (ii) N (L) = {0}; (iii) (L) = dim U ;

(iv) {u1 , . . . , un } is a basis of U L(u1 ), . . . , L(un ) is a basis of R(L). Proof. (i) (ii): Let u N (L). Hence L(u) = 0 = L(0). Since L is injective, we have u = 0. N (L) = {0}. (ii) (iii) dim U = (L) + (L) = (L) + 0 ( dim N (L) = (L) = 0)

= (L) = rank of L. (iii) (iv) From Theorem 3.4.2, R(L) = L(u1 ), . . . , L(un ) . Since dim R(L) = n, it follows that L(u1 ), . . . , L(un ) is a basis of R(L). (iv) (i) Let u1 , . . . , un be a basis of U . Suppose that L(u) = L(w ) where u, w U . Since u =
n i=1 ci ui ,

w=

n i=1

di ui for some ci , di F , we have

n n

L(u) = L
i=1 n

ci u i di u i
i=1

=
i=1 n

ci L(ui ) di L(ui ).
i=1

L(w ) = L Hence
n i=1 (ci

di )L(ui ) = 0. Since L(u1 ), . . . , L(un ) is a basis of R(L), it follows that

L(u1 ), . . . , L(un ) are linearly independent. Thus ci d i = 0 i = 1 , . . . , n ci = d i u=w L is injective. Theorem 3.4.7 Let L : U V be a linear transformation where dim U = dim V < . Then the following conditions are equivalent: 57 i = 1, . . . , n

(i) L is injective;

(ii) N (L) = {0};

(iii) (L) = dim V ;

(iv) {u1, . . . , un } is a basis of U

L(u1 ), . . . , L(un ) is a basis of V ;

(v) L is bijective (one-to-one and onto)

Denition 3.4.3 A bijective linear map L : U V is called an isomorphism from U to V.

Denition 3.4.4 Let U, V be vector spaces over a eld F . Then U is said to be isomorphic to V if there exists an isomorphism from U to V . In this case we write U =V.

Remark 3.4.1 An isomorphism f from U to V is invertible and f 1 : V U is also an isomorphism.

Example 3.4.8 Let U and V be 2 nite dimensional vector spaces over the same eld F . Then U = V dim U = dim V .

Remark 3.4.2 Every n-dimensional vector space over F is isomorphic to F n . (Here F n = {(a1 , . . . , an ) : ai F, i = 1, . . . , n}.)

Example 3.4.9 M2 (R) is isomorphic to R4 since the mapping : M2 (R) R4 dened by is an isomorphism. 58 a b c d = (a, b, c, d), a, b, c, d R

3.5

Matrices associated with linear transformations

Let L : U V be a linear transformation. Let S = {u1 , . . . , un } and T = {v1 , . . . , vm } be ordered bases of U and V respectively. Then L(u1 ) = a11 v1 + a21 v2 + + am1 vm L(u2 ) = a12 v1 + a22 v2 + + am2 vm . . . L(un ) = a1n v1 + a2n v2 + + amn vm for some aij F . a11 . . . a1n a 21 . . . a2n The matrix A = (aij ) = . Mm,n (F ) is called the matrix of L with . .. . . . . . am1 . . . amn respect to the bases S and T . The matrix is denoted by [L]T S . If U = V and S = T , then [L]T S is simply denoted by [L]S . [L]S is called the matrix of L with respect to the basis S . Note that [L]T S = ( [L(u1 )]T | [L(u2 )]T | | [L(un )]T ) where [L(uj ]T is the coordinate vector of L(uj ) with respect to the basis T . Example 3.5.1 Let L : R2 R2 be the linear mapping given by L((x, y )) = (x + y, 2x + 4y ), x, y R. Find (i) Find the matrix of L with respect to the basis S = {(1, 0), (0, 1)}; (ii) Find the matrix of L with respect to the basis T = {(1, 1), (1, 2)}; (iii) Find the transition matrix from T to S , [P ]S T or P ; (iv) Verify that [L]T = P 1 [L]S P . Solution: (i) L((1, 0)) = (1, 2) = 1(1, 0) + (2)(0, 1) L((0, 1)) = (1, 4) = 1(1, 0) + 4(0, 1) [L]S = 1 1 2 4 .

59

(ii) L((1, 1)) = (2, 2) = 2(1, 1) + 0(1, 2) L((1, 2)) = (3, 6) = 0(1, 1) + 3(1, 2) [L]T = (iii) (1, 1) = 1(1, 0) + 1(0, 1) (1, 2) = 1(1, 0) + 2(0, 1) [P ]S T = (iv) P [L]T = 1 1 1 2 1 1 2 0 0 3 1 1 1 2 = 2 3 2 6 = 2 3 2 6
1

2 0 0 3

1 1 1 2

[L]S P =

2 4

P [L]T = [L]S P,

i.e., [L]T = P

[L]S P .

Theorem 3.5.1 Let S and T be two ordered bases of a nite dimensional vector space U and L : U U be a linear map. Then [L]T = P 1 [L]S P where P is the transition matrix from the basis T to the basis S . (Note that we have [L]S [L]T .) Recall that a matrix A Mn (F ) is similar to a matrix B if an invertible matrix P Mn (F ) such that B = P 1 AP . In this case we write A B . Example 3.5.2 Show that A = matrices of a linear transformation. Solution: Let S = {(1, 0), (0, 1)}. Let L : R2 R2 be the linear map be dened by L((1, 0)) = 1(1, 0) + 2(0, 1) = (1, 2) L((0, 1)) = 0(1, 0) + 0(0, 1) = (0, 0) 60 1 0 2 0
S S S

1 0 0 0

= B by showing that A, B are

Then [L]S =

1 0 2 0

. Note that L((1, 2)) = (1, 2) = 1(1, 2) + 0(0, 1) L((0, 1)) = (0, 0) = 0(1, 2) + 0(0, 1) [L]T =
S

1 0 0 0

where T = {(1, 2), (0, 1)} 1 0 2 0

S

[L]S [L]T , i.e., 1 Example 3.5.3 Show that A = 2 4 3 3 L : R R such that [L]S = A and

1 0 0 0

0 0 1 0 S 0 0 B = 0 0 0 0 0 0 [L]T = B where S and

0 T are bases of R3 .

0 by nding a linear map

Let Hom(V, V ) be the set of all linear mappings from V to V where V is a vector space over a eld F . Dene two operations + and as follows: (f + g )(v ) = f (v ) + g (v ), v V, f, g Hom(V, V ); (f )(v ) = (f (v )), v V, F, f Hom(V, V ). It can be shown that f + g and f are linear maps from V to V and hence f + g Hom(V, V ), f Hom(V, V ). It can be shown that Hom(V, V ) is a vector space over F under the above 2 operations. The zero vector of Hom(V, V ) is the zero mapping 0 dened by 0(x) = 0 x V . For any f Hom(V, V ), let f : V V be dened by (f )(v ) = (f (v )), v V . Then f Hom(V, V ) and f + (f ) = (f ) + f = 0. Hom(V, V ) is called the vector space of all linear transformations on V . Theorem 3.5.2 Let V be an n-dimensional vector space over F . Then Hom(V, V ) = Mn (F ). Hints of Proof. Let v1 , . . . , vn be a basis of V . Let : Hom(V, V ) Mn (F ) be dened by (f ) = [f ]S where S = {v1 , . . . , vn }.

Then is a bijective linear map from Hom(V, V ) to Mn (F ). 61

Example 3.5.4 Let S = {1 = (1, 3), 2 = (3, 2)}. Find a linear map L : R2 R2 such 2 1 that [L]S = . 2 3 Solution: Let L : R2 R2 be the linear map such that L(1 ) = 21 22 = 2(1, 3) + (2)(3, 2) = (4, 2) L(2 ) = 1 + 32 = (1, 3) + 3(3, 2) = (8, 3) [L]S = 2 2 1 3 .

(x, y ) = c1 1 + c2 2 1 1 = (2x + 3y )1 + (3x y )2 (Why?) 7 7 L((x, y )) = L(c1 1 + c2 2 ) = c1 L(1 ) + c2 L(2 ) 1 1 = (2x + 3y )(4, 2) + (3x y )(8, 3) 7 7 1 1 = (8x 12y, 4x + 6y ) + (24x 8y, 9x 3y ) 7 7 1 = (32x 20y, 5x + 3y ). 7

3.6

Properties of similar matrices

(i) A A since A = I 1 AI , I is an identity matrix;
S S

Theorem 3.6.1 (ii) A B

S S

B A;
S

(iii) A B, B C

A C.

Similarity is an equivalence relation. Proof. (iii) AB BC

S S

B = P 1 AP for some invertible matrix P C = Q1 BQ for some invertible matrix Q

S

Hence C = Q1 (P 1 AP )Q = (Q1 P 1)A(P Q) = (P Q)1 A(P Q), i.e., A C . Remark 3.6.1 (ii) A B
S

(i) A B and A is invertible

S

A1 B 1 ;

Ak B k for any positive integer k ; 62

(iii) A B (iv) A B
S

(A) = (B ), (A) = (B ); |A| = |B | and tr(A) = tr(B ).

(Let C = (cij ) Mn (F ). Then the trace of C , denoted by tr C is c11 + c22 + + cnn .) (It is true that A, B Mn (F ), tr(AB ) = tr(BA).)

63

Chapter 4 Eigenvalues and Eigenvectors

4.1 Eigenvalues and eigenvectors

Denition 4.1.1 Let V be a vector space over a eld F . Let L : V V be a linear mapping. A vector v V, v = 0, is called an eigenvector of L if there exists F such that L(v ) = v. The scalar is called an eigenvalue of L and v is called the eigenvector of L corresponding to . Remark 4.1.1 Let v V be an eigenvector of L : V V . Then for any c F \{0}, cv is an eigenvector of L. Proof. Since v is an eigenvector of L, then L(v ) = v for some F . Hence, L(cv ) = cL(v ) = c(v ) = (cv ). cv is an eigenvector of L. (Note that cv = 0 since c = 0, v = 0.) Example 4.1.1 Let L : R2 R2 be the linear map dened by L((x, y )) = (3x, 8x y ). Then L((1, 2)) = (3, 6) = 3(1, 2) L((0, 1)) = (0, 1) = (1)(0, 1). Hence, (1, 2) is an eigenvector of L corresponding to the eigenvalue 3. And (0, 1) is an eigenvector of L corresponding to the eigenvalue -1. Denition 4.1.2 Let A Mn (F ). A nonzero vector v in Mn,1 (F ) is called an eigenvector of A if F such that Av = v . The scalar is called an eigenvalue of A and v is called the eigenvector corresponding to the eigenvalue . 64

Example 4.1.2 Let A =

8 1 3 0

and v1 =

1 2

, v2 =

0 1

. Then

Av1 =

1 2 0 1

8 1 3 0

3 6 0 1

=3

1 2

= 3v1 0 1

Av2 =

8 1

= (1)

= v2 .

v1 is an eigenvector of A corresponding to the eigenvalue 3. v2 is an eigenvector of A corresponding to the eigenvalue -1. . Note that A has 2 linearly independent eigenvectors v1 , v2 . Let P = (v1 . . v2 ) M2 (R), 1 0 i.e., P = . Then P is invertible. 2 1 . AP = A(v1 . . v2 ) = P 1AP =
S

1 0 2 1 = 3 0

8 1 3 0

6 1

2 1

6 1

0 1

A a diagonal matrix with diag(3, -1). Note that A = P DP 1 and we have An = (P DP 1)n = (P DP 1)(P DP 1) (P DP 1)
n times

= PD P

1 0 2 1 0

3n 0

0 (1)n .

2 1

3n

2.3n 2(1)n (1)n

Remark 4.1.2 Let L : V V be a linear mapping where S = {1 , . . . , n } is basis of V . Then L(c1 1 + + cn n ) = d1 1 + + dn n c1 d1 . . . . [L]S . = . . cn dn 65

(4.1)

It follows from (4.1) that v = c1 1 + + cn n V is an eigenvector of V corresponding to eigenvalue c1 . . . is an eigenvector of [L]S corresponding to eigenvalue . cn Theorem 4.1.1 Let L : V V be a linear mapping and S = {1 , . . . , n } be a basis of V . Then (i) L and [L]S have the same eigenvalues; (ii) c1 + cn n is an eigenvector of L corresponding to the eigenvalue 1 + c1 . . . is an eigenvector of [L]S corresponding to eigenvalue . cn Theorem 4.1.2 Let be an eigenvalue of L : V V . Let V = {v V : L(v ) = v }. Then V V . Proof. (i) L(0) = 0 0 V . (ii) Let v1 , v2 V , c F . Then L(v1 ) = v1 , L(v2 ) = v2 and hence L(v1 + v2 ) = L(v1 ) + L(v2 ) = v1 + v2 = (v1 + v2 ) v1 + v2 V L(cv1 ) = cL(v1 ) = c(v1 ) = (cv1 ) cv1 V V V. Denition 4.1.3 V in Theorem 4.1.2 is called the eigenspace of L corresponding to the eigenvalue . Note that V = the set of all eigenvectors corresponding to {0}. Theorem 4.1.3 Let A Mn (F ). Then F is an eigenvalue of A |A In | = 0. 66

Proof. is an eigenvalue of A v Mn,1 (F ), v = 0 such that Av = v v Mn,1 (F ), v = 0 such that A In = 0 x1 . . the system of homogeneous equations (A In ) . =0 xn has a nontrivial solution A In is not invertible |A In | = 0. Denition 4.1.4 Let A Mn (F ). Then (i) |A In | is called the characteristic polynomial of A. (ii) |A In | = 0 is called the characteristic equation of A.

Remark 4.1.3 Let A Mn (F ), A = (aij ). Then a11 |A In | = a21 . . . an1 is a polynomial of degree n in the variable . Remark 4.1.4 From Theorem 4.1.3, we see that = 0 is an eigenvalue of A Mn (F ) |A 0In | = 0 = |A| A is a singular matrix, i.e., A is not invertible. a12 a1n a2n . . .

a22 . .. . . . an2

ann

Theorem 4.1.4 If A Mn (F ) has n eigenvalues 1 , . . . , n , then (i) |A| = 1 n ; (ii) trA = 1 + + n . Proof. Note that |A In | = (1)n n + b1 n1 + + bn1 + bn = (1)n ( 1 ) ( n ) = f (). Then 67

(i) f (0) = |A 0In | = |A| = (1)n (1)n 1 n = 1 n ; (ii) The coecient of n1 in f () is equal to b1 . It is equal to (1)n1 (??) b1 = (1)n (1)(1 + + n )
n n i=1

aii . This

can be shown by expanding the determinant |A In | by using cofactors. From

= (1)
n

n1 i=1

aii

1 + + n =
i=1

aii = trA.

Theorem 4.1.5 The characteristic polynomials of two similar matrices A and B are the same. Proof. A B B = P 1 AP
S

for some invertible matrix P

|B In | = |P 1AP In | = |P 1 AP P 1 (In )P | = |P 1(A In )P | = |P 1 ||A In ||P | = |A In ||P 1||P | = |A In | ( |P 1| = char. poly. of A = char. poly. of B . Corollary 4.1.1 If A B , then A and B have the same eigenvalues.
S

1 ) |P |

Remark 4.1.5 A =

1 1

0 1 not similar to B . (Why?)

and B =

1 0 0 1

have the same eigenvalues 1, 1. But A is

Example 4.1.3 Let A = of A is |A I2 | =

3 1 1 3

. Then A is symmetric. The characteristic polynomial

3 1 1 3
2

1 0 0 1
2

3 1

1 3

= (3 ) 1 = 6 + 8 = ( 4)( 2). Hence 2, 4 are eigenvalues of A. 68

(i) = 2 3 1 1 3 x y =2 x y 3 x + y = 2x x + 3y = 2y t t x = y

Every eigenvector corresponding to = 2 is of the form (ii) = 4 3 1 1 3 x y =4 x y 3 x + y = 4x x + 3y = 4y

where t R\{0}.

t t

x=y

Every eigenvector corresponding to = 4 is of the form

where t R\{0}.

Take P =

1 1

. Then we have P 1 AP = diag (2, 4). Note that |A| = 8 = 4 2 = product of the eigenvalues, trA = 6 = 4 + 2 = sum of the eigenvalues.

Theorem 4.1.6 Let v1 be an eigenvector of A Mn (F ) corresponding to the eigenvalue 1 . Let v2 be an eigenvector of A Mn (F ) corresponding to the eigenvalue 2 . If 1 = 2 , then v1 , v2 are linearly independent. Proof. Suppose that c1 v1 + c2 v2 = 0. We shall show that c1 = c2 = 0. Then A(c1 v1 + c2 v2 ) = A0 = 0 c1 (Av1 ) + c2 (Av2 ) = 0 c1 1 v1 + c2 2 v2 = 0 c1 1 v1 + c2 1 v2 = 0 (multiply (4.2) by 1 ) c2 (2 1 )v2 = 0 ((4.3) (4.4)) Since 2 1 = 0, v2 = 0, it follows that c2 = 0. c1 v1 = 0 Thus, v1 , v2 are linearly independent. 69 c1 = 0 ( v1 = 0). (4.3) (4.4) (4.2)

Theorem 4.1.7 Let vi be an eigenvector of A Mn (F ) corresponding to the eigenvalue i , i = 1, . . . , k . If 1 , . . . , k are all distinct, then v1 , . . . , vk are linearly independent.

Denition 4.1.5 A matrix A Mn (F ) is called diagonalizable over F if P Mn (F ), P invertible, such that P 1 AP is diagonal.

Remark 4.1.6 (i) If D Mn (F ) is diagonal, then D is diagonalizable over F since I 1 DI = D . (ii) If A is diagonalizable over F , then At is also diagonalizable over F . (Exercise)

Remark 4.1.7 Let A Mm,n (F ) and B = (v1 | v2 | | vp ) Mn,p (F ) where vi is the ith column of B , i = 1, . . . , p. Then AB = A(v1 | v2 | | vp ) = (Av1 | Av2 | | Avp ). Hence Avi is the ith column of AB .

Example 4.1.4 a + 2b + 3c 4 a + 5b + 6c 7 a + 8b + 9c 1 4 7 a b c d e f 2 5 8 = d + 2e + 3f 4d + 5e + 6f 7d + 8e + 9f g + 2h + 3k 4 g + 5h + 6k 7 g + 8h + 9k 3 6 9 g h k
A (v1 |v2 |v3 ) Av1 Av2 Av3

Theorem 4.1.8 A Mn (F ) is diagonalizable over F A has n linearly independent eigenvectors in Mn,1 (F ). Proof. () Suppose that A has n linearly independent eigenvectors v1 , . . . , vn . Then Avi = i vi , i = 1, . . . , n for some i F . Let P = (v1 | v2 | | vn ). Then (P ) = dim (row space of P ) = dim (column space of A) = n since v1 , . . . , vn is a basis of the column space of A. P is invertible. We wish to show that P 1 AP = D , i.e., AP = P D where D is the diagonal matrix diag (1 , . . . , n ). 70

AP = A(v1 | v2 | | vn ) = (Av1 | Av2 | | Avn ) = (1 v1 | 2 v2 | | n vn ) 1 0 0 . . .. = (1 v1 | 2 v2 | | n vn ) . . P D = (v1 | v2 | | vn ) . . . 0 n AP = P D , P 1 (AP ) = P 1(P D ) = (P 1 P )D = In D = D . Hence A D , i.e., A is diagonalizable over F . Corollary 4.1.2 If A Mn (F ) has n distinct eigenvalues in F , then A is diagonalizable over F . Proof. A has n distinct eigenvalues A has n linearly independent eigenvectors in Mn,1 (F ) (Theorem 4.1.7). Thus, A is diagonalizable over F . 1 0
S

Remark 4.1.8 I2 =

is diagonalizable and has 2 same eigenvalues. This shows o 1 that the converse of Corollary 4.1.2 is false.

Example 4.1.5 Determine whether A = 1 2 1 0 Solution: |A I3 | = 1 1 0 2 0 2 1 3

0 0 2

1 is diagonalizable over R. 3

= ()(2 )(3 ) 2( 2) = ( 2)[3 2 2] = ( 2)2 ( 1).

A has eigenvalues 1, 2, 2. (i) = 1 0 0 2 x x 2z 1 2 1 y = y = x + 2y + z 1 0 3 z z x + 3z 71

x = 2z y = x + 2y + z

x = 2z y = x z = 2z z = z

(ii) = 2

z = x + 3z 2 2 t Eigenspace of = 1 is t : t R. So, let v1 = 1 . 1 t

0 0 2 x x 2z 1 2 1 y = 2 y = x + 2y + z 1 0 3 z z x + 3z 2x = 2z 2 y = x + 2y + z x = z

2 z = x + 3z t 0 1 Eigenspace of = 2 is s : s, t R. So, let v2 = 0 , v3 = 1. Then t 0 1 2 1 0 1 0 v1 , v2 , v3 are linearly independent since linearly independent 2 1 (Note: Let P = 0 1 1 1 i > j. a d e A= 0 b f 0 0 c a 0 0 Eigenvalues of A are a, b, c. 72 d 0 |A I3 | = b 1 = 1 = 0. Hence, A has 3 1 1 0 eigenvectors. A is diagonalizable over R. 0 1 1 . Then P AP = diag(1, 2, 2).) 0

Remark 4.1.9 A matrix A = (aij ) Mn (F ) is called upper triangular if aij = 0 for

is upper triangular and

e f c = (a )(b )(c ).

In general, if A = (aij ) Mn (F ) is upper triangular, then |A In | = (a11 )(a22 ) . . . (ann ) and hence a11 , a22 , . . . , ann are eigenvalues of A. 0 1 0 Example 4.1.6 Determine whether A = 0 0 1 is diagonalizable over R. 0 0 0 are eigenvalue of A. 1 0 x x 0 1 y = 0 y , 0 0 z z y 0 z = 0 , 0 0

Method 1 0, 0, 0 0 0 0

y=z=0

x Eigenspace of 0 is : x R . Hence, A does not have 3 linearly independent 0 0 over R.

S S

eigenvectors. ( Every two eigenvectors are linearly dependent.) A is not diagonalizable Method 2 If A diag(a, b, c), then a = b = c = 0 and we have A 0. 0A
S

A = P 1 0 P = 0

for some invertible matrix P , a contradiction. Thus, A is not diagonalizable over R. 2 1 0 Exercise: Determine whether 0 2 1 is diagonalizable over R. 0 0 2 Denition 4.1.6 A matrix A Mn (F ) is called idempotent if A2 = A. 1 0 0 is idempotent. It has eigenvalue 1, 1, 0. Example 4.1.7 A = 0 0 0 0 0 1 For any invertible matrix P M3 (R), P 1 AP is idempotent since (P 1 AP )2 = (P 1 AP )(P 1AP ) = P 1A2 P = P 1AP.

73

Theorem 4.1.9

(i) If is an eigenvalue of an idempotent matrix A, then = 0 or 1.

S

(ii) A Mn (F ) is idempotent A diag(1 , . . . , n ) where i {0, 1}, i = 1, . . . , n.

Theorem 4.1.10 Every real symmetric matrix is diagonalizable over R.

Remark 4.1.10 There exists a linear mapping T : R2 R2 which does not have any eigenvalue. Let T : R2 R2 be the following linear map. T ((x, y )) = x cos T has no eigenvector. Denition 4.1.7 Let L : V V be a linear transformation where dim V < . Then L is called diagonalizable over F (Here V is a eld F .) if a basis v1 , . . . , vn of V such that L(vi ) = i vi for some i F, i = 1, . . . , n, y sin , x sin + y cos 5 5 5 5 (rotation map).

i.e., if there exists a basis S such that [L]S is diagonal.

4.2

Matrix polynomials

Let f () = a0 + a1 + + an n , ai F , F is a eld. Let A Mn (F ). Then f (A) is dened by f (A) = a0 In + a1 A + + an An . (f (A) becomes a polynomial in A.) Example 4.2.1 Let A = polynomial of A. Then f (A) = A2 3A + 2I2 = 1 9 0 4 3 9 0 6 + 2 0 0 2 = (f () = 2 3 + 2) 0 0 0 0 . 1 3 0 2 . Let f () = (1 )(2 ) be the characteristic

74

Theorem 4.2.1 (Cayley-Hamilton Theorem) If f () is the characteristic polynomial of A, then f (A) = 0. 1 4 6 Example 4.2.2 Verify the Cayley-Hamilton Theorem for the matrix A = 0 2 5. 0 0 3 1 0 0 4 6 Solution: Char. poly. of A is f () = 2 0 5 3 = (1 )(2 )(3 ).

A satises its characteristic polynomial.

f (A) = (I3 A)(2I3 A)(3I3 A) 0 4 6 1 4 6 2 = 0 1 5 0 0 5 0 0 0 2 0 0 1 0 0 0 26 2 4 6 0 = 0 0 10 0 1 5 = 0 0 0 2 0 0 0 0 2 0 0

4 6 1 0 0 0 0

0 0 . 0 0

by using the Cayley-Hamilton Example 4.2.3 Find the inverse of A = 1 3 0 3 5 3 Theorem. Solution: Char. poly. of A is 2 f ( ) = 1 3 0 3 5 0 0 3 = 3 + 82 21 + 18. By Cayley-Hamilton Theorem, f (A) = A3 + 8A2 21A + 18I3 = 0 A(A2 + 8A 21I3 ) = 18I3 1 2 1 (A 8A + 21I3 ) = 6 18
7 9 1 2

= (2 )(3 )2

0
1 3

A1 =

5 9

0 .
1 3

75

Example 4.2.4 Let A =

1 2 3 4

. Using the Cayley-Hamilton Theorem to nd

(i) An for any positive integer n; (ii) A6 + 3A5 + I2 . 1 3 2 4

Solution: Char. poly. of A is f () = (i) Then

= 2 3 + 2. (4.5) ( f (1) = 0), ( f (2) = 0). (4.6) (4.7)

n = (2 3 + 2)q (x) + a + b 1n = f (1)q (1) + a + b = a + b 2n = f (2)q (2) + 2a + b = 2a + b

(4.7) - (4.6) gives us that a = 2n 1, b = 1 a = 2 2n . From (4.5), An = f (A)q (A) + aA + bI2 = aA + bI2 =a 1 2 3 4 a + b 3a +b 2a 4a + b 1 0 0 1 where a = 2n 1, b = 2 2n . ( f (A) = 0)

(ii) For = 1 =2

6 + 35 + 1 = (2 3 + 2)q () + a + b 5=a+b 26 + 3 25 + 1 = 161 = 2a + b

(4.8) (4.9) (4.10)

(4.10) - (4.9) gives us that a = 161 5 = 156, b = 5 a = 151. From (4.8) A6 + 3A5 + I2 = f (A)q (A) + aA + bI2 = aA + bI2 = 156 1 2 3 4 151 1 0 0 1 = 307 312 468 473 . ( f (A) = 0)

Example 4.2.5 Let A =

1 0

. Find A10 + 3A9 + I2 . 76

Solution: Char. poly. of A is f () = Hence = 1 is a repeated eigenvalue.

2 1

= 2 2 + 1 = ( 1)2 .

10 + 39 + 1 = ( 1)2 q () + a + b Put = 1, we get 5 = 0 + a + b Dierentiating (4.11), 109 + 278 = f ()q () + f ()q () + a. Put = 1, 37 = f (1)q (1) + f (1)q (1) + a =a ( f () = 2( 1) f (1) = 0)

(4.11) (4.12)

From (4.12), we have b = 5 a = 32. And it follows from (4.11), A10 + 3A9 + I2 = f (A)q (A) + aA + bI2 = aA + bI2 = 37 2 1 1 0 42 37 + (32) 1 0 0 1 ( f (A) = 0)

37 32

77