1 Linear Systems of Equations

1.1 Linear Systems

The subject of algebra arose from studying equations.
linear equation in n unknowns: a
1
x
1
+ + a
n
x
n
= b. 2x + 3y = 6 line, 2x + 3y + 4z = 12 plane, 2x
2
+ 3y = 10
nonlinear
system of m linear equations in n unknowns: a
11
x
1
+ +a
1n
x
n
= b
1
(m = n = 2) inconsistent (no solution): 2x 3y = 2 and 6x + 9y = 7; independent consistent: 2x 3y = 2
and 4x +y = 24; dependent consistent: 2x 3y = 2 and 4x 6y = 4. Graphs
How do we handle more general cases when (m, n) = (2, 2)?
1.2 Matrices
Def An m n matrix is a rectangular array of numbers arranged in m horizontal rows and n vertical columns.
The entries in the matrix are called the elements of the matrix.
Ex Give examples, and illustrate a
ij
and b
ij
Def A 1 n matrix is called a row vector. An n 1 matrix is called a column vector.
Ex Give examples
Def Two mn matrices A = [a
ij
] and B = [b
ij
] are equal if they agree entry by entry.
Ex If
_
a +b c +d
c d a b
_
=
_
4 6
10 2
_
, nd the variables.
Def (Matrix Addition) If A = [a
ij
] and B = [b
ij
] are both mn matrices, then the sum A+B is an mn matrix
C = [c
ij
] dened by c
ij
= a
ij
+b
i
for all pairs of indices (i, j).
Ex A =
_
1 2 3
2 1 4
_
and B =
_
0 2 1
1 3 4
_
Def (Scalar Multiplication) If A = [a
ij
] is an mn matrix and r is a real number, then the scalar multiple of A
by r, rA, is the mn matrix C = [c
ij
], where c
ij
= ra
ij
for all pairs of indices (i, j).
Ex r = 2 and A
We will refer to numbers as scalars.
We write A+ (1)B as AB. The mn zero matrix O is the mn matrix whose entries are all 0.
Ex AB.
Thm If A, B and C are mn matrices, O is the mn zero matrix, and r and s are scalars, then A+B = B+A,
A+ (B +C) = (A+B) +C, AA = O, A+O = A, (r +s)A = rA+sA.
Ex Find the matrices A and B such that A+
_
3 1
4 4
_
= B
_
3 1
4 4
_
and 2A+
_
0 3
5 0
_
= B+
_
3 9
14 12
_
=.
MTAP - Introduction to Linear Algebra 1 2nd Semester 2013-2014 (R. Eden)
Def If A = [a
ij
] is an mn matrix, then the transpose of A, A
T
= [c
ij
], is the n m matrix dened by c
ij
= a
ji
.
Ex A
T
, transpose of row and column vectors
Thm (A
T
)
T
= A, (rA)
T
= rA
T
, (A+B)
T
= A
T
+B
T
.
Def If A = [a
ij
] is an mn matrix and x = [x
i
] is an column vector with n entries, the product Ax is an n-column
vector y = [y
i
] dened by y
i
= a
i1
x
1
+ +a
in
x
n
=

n
j=1
a
ij
x
j
.
Ex A =
_
1 2 1
3 1 4
_
, p =
_
2 4 2

T
and q =
_
5 3 1

T
. Find Ap, Aq and A(p +q).
Thm A(p +q) = Ap +Aq
1.3 Reduced Row Echelon Form and Row Operations
Go back to system of equations, and write as Ax = b. Coecient matrix A and augmented matrix [A|b].
Ex x + 2y + 3z = 9, 2x y +z = 8, 3x z = 3
Ex x +y +z = 1, x y +z = 0
Def A matrix is in row-echelon form (REF) if it satises all of the following.
1. All zero rows, if there are any, are at the bottom of the matrix.
2. The rst nonzero entry from the left of a (nonzero) row is a 1. This entry is called the leading 1 of its row.
3. For each nonzero row, the leading one appears to the right and below any leading ones in preceding rows.
Ex A =
_

_
1 5 2 7
0 1 2 10
0 0 0 1
0 0 0 0
0 0 0 0
_

_
, B =
_
_
1 1 7 5
0 2 5 0
0 0 1 9
_
_
, C =
_
_
1 1 7 5
0 1 1 2
0 0 1 1
_
_
Ex Linear System with augmented matrix in REF:
_
_
1 2 3 9
0 1 1 2
0 0 1 3
_
_
. Solve using back substitution.
Def A matrix is in reduced row-echelon form (RREF) if it is in row-echelon form, and furthermore, if a column
contains a leading one, all other entries in that column are 0.
Ex Give examples
Def An elementary (row) operation on a matrix A is any one of the following operations:
1. Interchange/switch any two rows.
2. Multiple a row by a nonzero number.
3. Add a multiple of one row to another.
MTAP - Introduction to Linear Algebra 2 2nd Semester 2013-2014 (R. Eden)
Def A matrix B is equivalent to a matrix A if B can be obtained by applying a nite sequence of elementary
operations on A.
Thm Let Ax = b and Cx = d be two linear systems, each of m equations in n unknowns. If the augmented
matrices [A|b] and [C|d] are equivalent, then the linear systems have exactly the same solutions.
Strategy: to solve Ax = b, nd an equivalent augmented matrix in REF or RREF (by using the row operations),
and nd the solutions using the latter.
Ex Convert to REF, and RREF:
_

_
2 1 1 3
1 1 2 1
4 6 7 1
2 0 1 3
_

_
,
_
_
3 2 5 2
1 1 1 1
1 0 3 4
_
_
,
_
_
3 1 22
1 5 2
2 4 24
_
_
,
_
_
2 1 3 4
1 2 1 3
1 5 0 5
_
_
,
_
_
2 1 3
3 1 2
2 2 1
_
_
Thm Let A be an mn matrix. All REF matrices that are row-equivalent to A have the same number of nonzero
rows.
Def The rank of a matrix A is the number of nonzero rows in its REF
Gaussian elimination and Gauss-Jordan elimination
Ex x + 2y z = 1, 2x + 5y z = 3, x + 3y + 2z = 6, (5, 1, 2)
Ex 5x 6y +z = 4, 2x 3y +z = 1, 4x 3y z = 5, (2 +t, 1 +t, t)
free variable: a variable we choose to specify; bound variable: the remaining variables
Ex x
1
2x
2
+ 2x
3
x
4
= 3, 3x
1
+x
2
+ 6x
3
+ 11x
4
= 16, 2x
1
x
2
+ 4x
3
+ 4x
4
= 9, (5 2s 3t, 1 2t, s, t)
Choose as free variables those variables that do not correspond to a leading 1 in the REF of A. Number of free
variables = n rank(A
#
).
Ex x
1
+2x
2
+x
3
+x
4
2x
5
= 3, x
3
+4x
4
3x
5
= 2, 2x
1
+4x
2
x
3
10x
4
+5x
5
= 0, (12r+3st, r, 24s+3t, s, t)
Ex x
1
+x
2
x
3
+x
4
= 1, 2x
1
+ 3x
2
+x
3
= 4, 3x
1
+ 5x
2
+ 3x
3
x
4
= 5; inconsistent
Thm Consider the mn linear system Ax = b. Let A
#
be the augmented matrix [A|b]
1. If rank(A) < rank(A
#
), the system is inconsistent.
2. If rank(A) = rank(A
#
), the system is consistent.
(a) There is a unique solution i rank(A) = n.
(b) There are innitely many solutions i rank(A) < n.
Ex x +y z = 2, x + 2y +z = 3, x +y + (a
2
5)z = a; no solution, unique solution, innitely many solutions
Ex x +ay +a
2
z = 1, x +ay +abz = a, bx +a
2
y +a
2
bz = a
2
b
Thm Suppose a consistent system of linear equations has m equations in n variables. If n > m, then the system
has innitely many solutions.
MTAP - Introduction to Linear Algebra 3 2nd Semester 2013-2014 (R. Eden)
The linear system Ax = b is said to be a homogeneous system if b = 0.
Thm The homogeneous linear system Ax = 0 is consistent.
Ex x + 2y + 2z = 0, 2x + 5y + 7z = 0, 3x + 6y + 6z = 0; innitely many
Ex x + 2y + 2z = 0, 2x + 5y + 7z = 0, 3x + 6y + 6z = 0; only trivial
Ex x
1
+ 2x
2
+x
3
+ 2x
4
= 0, 2x
1
+ 4x
2
+x
3
+ 3x
4
= 0, 3x
1
+ 6x
2
+x
3
+ 4x
4
= 0
Ex 2x +y +z = 0, 4x + 2y +z = 0, 6x + 3y +z = 0, 8x + 5y +z = 0; only trivial
Cor In a homogeneous linear system Ax = 0, if n > m, there are innitely many (nontrivial) solutions.
Ex 4x
3
+ 2y
2
+ 3z = 12, x
3
y
3
+z = 2, 3x
3
+y
2
z = 2.
If x = v is a solution to Ax = 0, then so is rv for any scalar r. If x = v and x = w are solutions, so is v +w. Thus,
rv +sw is a solution for any scalars r and s.
If x = v is a solution of Ax = 0 and x = w is a solution of Ax = b, then for any scalar r, w + rv is a solution of
Ax = b.
Ex Let A =
_

_
1 1 2 2 1
2 2 4 4 3
2 2 4 4 2
3 5 8 6 5
_

_
and b =
_
1 1 2 3

T
. Solve Ax = 0 and Ax = b. (two free variables)
Ex Balance the equation NaOH +H
2
SO
4
Na
2
SO
4
+H
2
O
2 Matrices
2.1 Matrix Multiplication
Recall how to multiply an mn matrix A = [a
ij
] with a column vector x having n entries. Use this to multiply
an n-row vector with an n-column vector. We will use R
n
to denote the set of n-(column) vectors.
Def The dot product, or inner product, of two vectors a = [a
i
] and b = [b
i
] in R
n
is a b =

n
i=1
a
i
b
i
.
Ex u =
_
1 2 3 4

and v =
_
2 3 2 1

Ex An instructor assigns the weights 10%, 25%, 25%, 40% to quizzes, two long exams, and the nal exam. A
student scored 78, 84, 62, 85, respectively. Find the course average: 78.3.
Def If A = [a
ij
] is an mp matrix and B = [b
ij
] is a p n matrix, then the product of A and B, denoted AB, is
the mn matrix C = [c
ij
] dened by
c
ij
= a
i1
b
1j
+a
i2
b
2j
+ +a
ip
b
pj
=
n

k=1
a
ik
b
kj
.
1. The (i, j)-element of AB is the dot product of row i from A, and column j from B.
2. Emphasize compatibility of sizes for multiplication.
MTAP - Introduction to Linear Algebra 4 2nd Semester 2013-2014 (R. Eden)
Ex A =
_
1 2 1
3 1 4
_
, B =
_
_
2 5
4 3
2 1
_
_
Ex A =
_
_
1 2 3
4 2 1
0 1 2
_
_
, B =
_
_
1 4
3 1
2 2
_
_
Ex A =
_
1 2
1 3
_
, B =
_
2 1
0 1
_
Matrix multiplication is not commutative. If AB and BA are of the same size, they may or may not be equal.
Also, u v = u
T
v.
Ex Let A and B be the matrices below, where a
ij
is the amount of pesticide i (mg) absorbed by plant j, and b
ij
is the number of plants of type i that a herbivore of type j eats per month. How much pesticide of type 3 does a
herbivore of type 2 eat per month?
A =
_
_
2 3 4 3
3 2 2 5
4 1 6 4
_
_
B =
_

_
20 12 8
28 15 15
30 12 10
40 16 20
_

_
Ex Let A =
_
cos sin
sin cos
_
. Find A
2
and A
3
. Conjecture and prove a simple expression for A
k
, k a positive
integer.
Ex Relate (AB)
T
with B
T
A
T
: A =
_
1 2 3
2 0 1
_
and B =
_
3 1 2
3 2 1
_
.
Thm Assume that A, B, and C are matrices of the appropriate sizes for the indicated multiplication.
1. A(BC) = (AB)C
2. (A+B)C = AC +BC
3. C(A+B) = CA+CB
4. (AB)
T
= B
T
A
T
Ex A =
_
1 2
2 4
_
and B =
_
4 6
2 3
_
: AB = 0.
Ex A =
_
1 2
2 4
_
, B =
_
2 1
3 2
_
, C =
_
2 7
5 1
_
: AB = AC
1. AB need not equal BA.
2. AB may be the zero matrix with A = O and B = O.
3. AB may equal AC with B = C.
Given: A and B below.
A =
_

_
a
11
a
12
a
13
a
14
a
15
a
21
a
22
a
23
a
24
a
25
a
31
a
32
a
33
a
34
a
35
a
41
a
42
a
43
a
44
a
45
_

_
=
_
A
11
A
12
A
13
A
21
A
22
A
23
_
B =
_

_
b
11
b
12
b
13
b
14
b
21
b
22
b
23
b
24
b
31
b
32
b
33
b
34
b
41
b
42
b
43
b
44
b
51
b
52
b
53
b
54
_

_
=
_
_
B
11
B
12
B
21
B
22
B
31
A
32
_
_
MTAP - Introduction to Linear Algebra 5 2nd Semester 2013-2014 (R. Eden)
Then C = AB can be determined by block multiplication.
Ex A =
_

_
1 0 1 0
0 2 3 1
2 0 4 0
0 1 0 3
_

_
and B =
_

_
2 0 0 1 1 1
0 1 1 1 2 2
1 3 0 0 1 0
3 1 2 1 0 01
_

_
Ex AB = A
_
B
1
B
2
B
p1
B
p

=
_
AB
1
AB
2
AB
p1
AB
p

2.2 Nonsingular Matrices and Other Matrices

A matrix with the same number of rows as columns is called a square matrix. The nn identity matrix I
n
= [a
ij
]
is the diagonal matrix where a
ii
= 1 and a
ij
= 0 if i = j.
Thm For any mn matrix A, AI
n
= A and I
m
A = A.
Def An n n matrix A is called nonsingular, or invertible, if there exists an n n matrix B such that AB =
BA = I
n
; such a B is called an inverse of A.
Thm If A and B are n n with AB = I
n
, then BA = I
n
. (Proof is postponed to a future time.)
Thm The inverse of a matrix, if it exists, is unique.
We write THE inverse of a nonsingular matrix A as A
1
: AA
1
= A
1
A = I
n
. Also, to verify that B is the
inverse of A, we need verify only one of AB = I
n
or BA = I
n
.
Ex A =
_
_
1 1 3
0 1 2
3 5 1
_
_
and B =
_
_
11
14

8
7
1
14

3
7
5
7
1
7
3
14
1
7

1
14
_
_
Thm Let A and B be nonsingular n n matrices.
1. AB is nonsingular and (AB)
1
= B
1
A
1
.
2. A
1
is nonsingular and (A
1
)
1
= A.
3. A
T
is nonsingular and (A
T
)
1
= (A
1
)
T
.
Cor If A
1
, A
2
, . . . , A
r
are n n nonsingular matrices, then A
1
A
2
A
r
is nonsingular and (A
1
A
2
A
r
)
1
=
A
1
r
A
1
r1
A
1
1
.
Ex A =
_
3 2
1 3
_
, A
1
=
1
7
_
3 2
1 3
_
, B =
_
2 7
3 2
_
, B
1
=
1
25
_
2 7
3 2
_
Is O
n
nonsingular? If A is nonsingular, is rA nonsingular where r = 0 is a scalar? If A and B are nonsingular, is
A+B nonsingular? What happens if A is nonsingular and AB = AC? If AB = 0?
Thm If A is nonsingular, the system of equations Ax = b has the unique solution x = A
1
b. The homogeneous
system Ax = b has only the trivial solution x = 0.
Ex Solve x +y + 3z = 2, y + 2z = 1, 3x + 5y z = 4. Then try for b = (2, 4, 0).
Ex Is A =
_
1 2
3 4
_
nonsingular? If yes, nd A
1
=
_
a b
c d
_
. Try for B =
_
1 2
2 4
_
.
MTAP - Introduction to Linear Algebra 6 2nd Semester 2013-2014 (R. Eden)
Thm An n n matrix A is nonsingular i rank(A) = n.
Proof: Suppose rank(A) = n. Let x
i
(vector) be the unique solution of Ax = e
i
. Let X =
_
x
1
x
2
x
n1
x
n

.
Then AX = I
n
. Next, we show XA = I
n
. From AX = I
n
, A(XA I
n
) = 0
n
with XA I
n
= [y
1
y
2
y
n
]. Then
Ay
i
= 0, with only the trivial solution since rank(A) = n. XAI
n
= 0
n
; we are done!
Cor Let A be an n n matrix. If Ax = b has a unique solution, then A
1
exists.
Thus, if A is square, Ax = b has a unique solution i A
1
exists. Also, a square matrix A is nonsingular i its
RREF is I
n
.
Thm Let A and B be n n matrices. If AB = I
n
, then both A and B are nonsingular and B = A
1
.
Proof: Claim: rank(A) = n. Otherwise, suppose rank(A) < n. Let A

be RREF of A and b

a vector with nonzero

last entry so A

x = b

is inconsistent. For some b, [A|b] is row equivalent to [A

|b

] so Ax = b is inconsistent.
However, x = Bb is a solution; contradiction.
Cor Let A and B be n n matrices. If AB is nonsingular, then both A and B are nonsingular.
Proof: Let C = B(AB)
1
so AC = I
n
so A is nonsingular. Let D = (AB)
1
A so CB = I
n
.
Let A be a square n n matrix. The following statements are equivalent.
1. A is nonsingular.
2. rank(A) = n.
3. A in RREF is I
n
.
4. Ax = b has a unique solution for any b: x = A
1
b.
5. Ax = 0 has only the trivial solution.
Suppose A is nonsingular so rank(A) = n. How do we nd its inverse, the matrix B such that AB = I
n
, i.e.,
AB = A
_
B
1
B
2
B
n1
B
n

=
_
AB
1
AB
2
AB
n1
AB
n

=
_
e
1
e
2
e
n1
e
n

Strategy: [A|I
n
] [I
n
|A
1
].
Ex Find the inverses of the following, if they are nonsingular:
_
1 3
2 6
_
,
_
_
3 1 2
2 1 2
1 2 2
_
_
,
_
_
1 2 3
1 2 1
5 2 3
_
_
(singular)
Ex C
T
B
1
A
1
=
_
2 1
1 1
_
, (A
T
)
1
=
_
2 1
0 1
_
and B =
_
3 1
1 2
_
. Find C.
Def A square matrix A is said to be
1. symmetric if A
T
= A: a
ij
= a
ji
2. skew symmetric if A
T
= A: a
ij
= a
ji
3. upper triangular if a
ij
= 0 for i > j
4. lower triangular if a
ij
= 0 for i < j.
Ex A and B are 3 3 upper triangular. Find A+B and AB.
Miscellaneous
MTAP - Introduction to Linear Algebra 7 2nd Semester 2013-2014 (R. Eden)
1. If A is skew-symmetric, then the main diagonal elements of A are 0.
2. For any mn matrix A, AA
T
and A
T
A are symmetric.
3. If A and B are symmetric/skew/upper/lower, so is A+B.
4. If A and B are upper/lower, so is AB.
5. If A is upper, A
T
is lower.
6. For any square matrix, A+A
T
is symmetric and AA
T
is skew symmetric.
7. Any square matrix A has a unique decomposition S +K where S is symmetric and K is skew.
If A = [a
ij
] is an n n matrix, the trace of A, Tr(A), is the sum of all elements on the main diagonal of A:
Tr(A) =

n
i=1
a
ii
.
Thm Properties of Trace
1. Tr(cA) = c Tr(A)
2. Tr(A+B) = Tr(A) + Tr(B)
3. Tr(A
T
) = Tr(A)
4. Tr(AB) = Tr(BA)
5. Tr(A
T
A) 0
Ex Show that there are no 2 2 matrices A and B such that AB BA = I
2
.
3 Determinants
3.1 Denition
Consider a permutation j
1
j
2
j
n
of S
n
= {1, 2, . . . , n}. The permutation is said to have an inversion if a larger
integer j
r
precedes a smaller one, j
s
. A permutation is called even or odd, if the total number of inversions is even
or odd.
Ex permutations of S
1
, S
2
and S
3
. Half the permutations are odd, half are even. Then parity of 52134 and 42135.
Def Let A = [a
ij
] be an n n matrix. The determinant of A, denoted det (A) or |A|, is dened by det(A) =

a
1j
1
a
2j
2
a
njn
where the summation is taken over all permutations j
1
j
2
j
n
of S
n
. The sign is taken as +
or according to whether the permutation is even or odd.
Each term is a product of n numbers, with exactly one entry from each row and exactly one entry from each
column. There are n! terms in the sum.
Develop determinant formulas for when n = 1, 2, 3. Warning: the diagonal computations do not apply when n 4.
Ex A =
_
_
1 2 3
2 1 3
3 1 2
_
_
,
_

_
0 0 2 0
0 3 0 0
6 0 0 0
0 0 0 5
_

_
Geometric Interpretation
1. P
1
P
2
P
3
with P
i
= (x
i
, y
i
) has area
1
2
| det (A) | where A =
_
_
x
1
y
1
1
x
2
y
2
1
x
3
y
3
1
_
_
.
MTAP - Introduction to Linear Algebra 8 2nd Semester 2013-2014 (R. Eden)
2. A parallelogram has sides determined by vectors < a
1
, a
2
> and < b
1
, b
2
>. The area is | det (A) | where
A =
_
a
1
a
2
b
1
b
2
_
.
3. A parallelepiped has sides determined by < a
1
, a
2
, a
3
> up to < c
1
, c
2
, c
3
>. The volume is | det (A) | where
A =
_
_
a
1
a
2
a
3
b
1
b
2
b
3
c
1
c
2
c
3
_
_
.
4. P
1
, P
2
and P
3
are collinear if and only if the determinant is 0.
5. The equation of the line through (x
1
, y
1
) and (x
2
, y
2
) is

x y 1
x
1
y
1
1
x
2
y
2
1

= 0.
Ex Triangle (1, 4), (3, 1), (2, 6)
Ex Find k so (1, 9), (k, 3) and (k + 3, 0) are collinear.
Ex Find the volume of the parallelepiped formed by the vectors (3, 2, 1), (1, 3, 0) and (2, 2, 5). Answer: 47
3.2 Properties
Thm det (A) = det
_
A
T
_
Thm If B is obtained from A by
1. switching two distinct rows of A, then det (A) = det (B),
2. multiplying a row of A by a constant k, then det (B) = k det (A),
3. adding a multiple of a row of A to a dierent row of A, then det (A) = det (B).
Cor If c is a scalar and A is n, then det (cA) = c
n
det (A).
Thm If two rows (columns) of A are equal, then det (A) = 0.
Thm If a row (column) equal consist entirely of zeros, then det (A) = 0.
Thm If A is upper or lower triangular, then its determinant is the product of all entries in the main diagonal.
Ex Evaluate

2 1 3 7
1 2 4 3
3 4 2 1
2 2 8 4

Thm A matrix A is nonsingular i det (A) = 0.

Proof: Let A be the RREF; det (A) is thus a nonzero multiple of det (A).
Ex Are the following nonsingular? A =
_
2 1
6 3
_
, B =
_
_
1 2 3
5 2 1
8 2 5
_
_
Ex Determine k so that the following are nonsingular.
_
_
k 1 2
0 1 1
1 0 k
_
_
.
_

_
k 0 0 1
0 k 0 0
0 0 k 0
1 0 0 k
_

_
MTAP - Introduction to Linear Algebra 9 2nd Semester 2013-2014 (R. Eden)
Cor The n n system Ax = b has a unique solution i det (A) = 0.
Cor The n n homogeneous system Ax = 0 has an innite number of solutions i det (A) = 0, and therefore has
just the trivial solution i det (A) = 0.
Ex Let A be a 4 4 matrix with det (A) = 2.
1. Describe the solutions of Ax = 0.
2. What is the RREF of A?
3. Can Ax = b have more than one solution?
4. Does A
1
exist?
Ex Evaluate a determinant to nd out if there are nontrivial solutions to Ax = 0 where A =
_

_
1 2 0 1
0 1 2 3
0 0 1 2
0 1 2 1
_

_
.
Ex Determine all k for which x+2y +z = kx, 2x+y +z = ky, x+y +2z = kz, has an innite number of solutions.
Thm det (AB) = det (A) det (B)
Ex A =
_
1 1
2 3
_
and B =
_
1 2
2 4
_
Cor If A is nonsingular, det
_
A
1
_
= 1/ det (A).
Ex Show that if A = A
1
, then det (A) = 1. What if A
T
= A
1
?
Ex Show that if A is an n n skew-symmetric matrix with n odd, then A is singular.
Ex If AB is nonsingular, explain using determinants why A and B are both nonsingular.
Ex If A
2
= A, what are the possible values of det (A)?
Ex If S is nonsingular, show that det
_
S
1
AS
_
= det (A).
Ex Without expanding, show that

1 x x
2
1 y y
2
1 z z
2

= (y z)(z x)(x y).

3.2 Cofactor Expansion
Def Let A = [a
ij
] be a square matrix. The cofactor A
ij
of a
ij
is dened as A
ij
= (1)
i+j
det (M
ij
), where M
ij
is
the submatrix of A obtained by deleting the ith row and jth column of A.
Ex A =
_
_
2 1 3
1 4 2
3 1 5
_
_
: A
11
= 22, A
23
= 1, A
31
= 14.
Thm Let A = [a
ij
] be an n n matrix. Then det (A) = a
i1
A
i1
+ a
i2
A
i2
+ + a
in
A
in
(expansion along the ith
row) and det (A) = a
1j
A
1j
+a
2j
A
2j
+ +a
nj
A
nj
(expansion along the jth column).
MTAP - Introduction to Linear Algebra 10 2nd Semester 2013-2014 (R. Eden)
Ex A =
_
_
2 3 4
1 1 1
6 3 0
_
_
, row 1, then column 3
Ex A =
_

_
0 3 1 0
5 0 8 2
7 2 5 4
6 1 7 0
_

_
, B =
_

_
2 1 8 6
1 4 1 3
1 2 1 4
1 3 1 2
_

_
(row operations and cofactors), C =
_

_
2 0 1 3 0
0 3 0 1 2
0 1 3 0 4
1 0 1 1 0
3 0 2 0 5
_

_
Def If every element in a square matrix A is replaced by its cofactor, the resulting matrix is called the matrix of
cofactors. The transpose of this matrix is called the adjoint of A and is denoted by adj (A), i.e., adj (A)
ij
= C
ji
Ex A =
_
_
3 2 1
5 6 2
1 0 3
_
_
Thm If det(A) = 0, then A
1
=
1
det(A)
adj (A).
Ex
_

_
0 2 1 3
2 1 3 4
2 1 5 2
0 1 0 2
_

_
,
_
_
4 2 2
0 1 2
1 0 3
_
_
,
_
a b
c d
_
Thm (Cramers Rule). Consider the linear system Ax = b where A is nn. If det (A) = 0, then x
1
=
det(A
1
)
det(A)
, x
2
=
det(A
2
)
det(A)
, . . . , x
n
=
det(An)
det(A)
, where A
i
is the matrix obtained from A by replacing the ith column of A by b.
Proof: Kolman (pdf page 183)
Ex 2x + 3y z = 1, x + 2y z = 4, 2x y +z = 3
Ex 2x +y +z = 6, 3x + 2y 2z = 2, x +y + 2z = 4
3.4 Matrix Transformations
Let A be m n, and u an n-vector, so Au is an m-vector. A function f mapping R
n
into R
m
is denoted by
f : R
n
R
m
. A matrix transformation is a function f : R
n
R
m
dened by f(u) = Au. The vector f(u) is
called the image of u, and the set of all images of vectors in R
n
is called the range of f.
Ex f(u) =
_
1 2 0
1 1 1
_
. u =
_
_
1
0
1
_
_
, u =
_
_
0
1
3
_
_
Ex A =
_
1 2
2 3
_
. Is w =
_
4
1
_
in the range of f(u) = Au?
Ex A =
_
1 0
0 1
_
acting on u =
_
x
y
_
: reection w.r.t. x-axis. How about w.r.t. y-axis?
Ex A =
_
_
1 0 0
0 1 0
0 0 0
_
_
: projection into the xy-plane. Innitely many 3-vectors have the same image.
MTAP - Introduction to Linear Algebra 11 2nd Semester 2013-2014 (R. Eden)
Ex A =
_
r 0
0 r
_
: dilation if r > 1, contraction if 0 < r < 1.
Ex
_
cos sin
sin cos
_
: rotation by the angle counterclockwise
Ex
_
0 1
1 0
_
; apply on
_
4
1
_
,
_
1
3
_
, and
_
6
2
_
; reection w.r.t. y = x
Ex
_
1 k
0 1
_
; apply on (0, 0), (0, 2), (4, 0) and (4, 2) for k = 2 and k = 3; shear in the x-direction
4 Eigenvalues and Eigenvectors
4.1 Eigenvalues and Eigenvectors
Def Let A be a square matrix. A number is said to be an eigenvalue of A if there exists a nonzero vector x s.t.
Ax = x. Every nonzero vector x satisfying this equation is called an eigenvector of A associated with .
Ex A =
_
5 4
8 7
_
,
1
= 3, x
1
=
_
1
2
_
,
2
= 1, x
2
=
_
4
4
_
Ex Which of the following are eigenvectors of A =
_
2 1
2 1
_
?
_
1
1
_
,
_
1
2
_
,
_
1
2
_
If we do not require that x be nonzero, then every number would be an eigenvalue, since A(0) = (0); such a
denition would be of no interest. This is why we insist (in the denition) that eigenvectors be nonzero.
Motivate: (AI)x = 0 has nontrivial solutions x i det (AI) = 0.
1. Find all scalars such that det (AI) = 0. These are the eigenvalues of A.
2. For an eigenvalue , solve the homogeneous system (A I)x = 0 for x to determine the associated eigen-
vectors.
Ex Find all eigenvalues and associated eigenvectors of A =
_
5 4
8 7
_
.
Given the n n matrix A, the degree n polynomial p() = det (AI) is called the characteristic polynomial of
A. The eigenvalues of A are the roots of p() = 0.
Ex A =
_
_
5 12 6
3 10 6
3 12 8
_
_
; (expand on row 1) p() = ( 2)
2
( + 1)
How many eigenvectors are associated with an eigenvalue? Innitely many. Talk about scalar multiples and sums
of eigenvectors.
Ex Let A =
_
1 0
0 1
_
. Describe the matrix transformation f(x) = Ax. By arguing geometrically, determine all
eigenvalues and eigenvectors of A.
Ex A =
_
2 6
3 4
_
has eigenvalues 1 3i.
MTAP - Introduction to Linear Algebra 12 2nd Semester 2013-2014 (R. Eden)
Ex What are the eigenvalues of A =
_
_
1 0 0
2 3 0
2 2 4
_
_
? (lower triangular).
Ex What are the eigenvalues of an upper/lower triangular matrix? The diagonal entries.
Ex Let A, , x be associated with each other. Consider A
n
x for n a positive integer.
Ex If A =
_
5 4
8 7
_
, nd A
5
v where v =
_
4
8
_
. A
10
w where v =
_
2
2
_
. A
5
z where z =
_
2
3
_
.
Ex Prove that A and A
T
have the same eigenvalues.
Ex Prove that A is singular i 0 is an eigenvalue.
Ex Eigenvalues of A and A
1
: 0 is not an eigenvalue. Consider A
n
x for n any integer.
Ex Prove that the determinant equals the product of the eigenvalues (counting multiplicity).
Ex The Cayley-Hamilton Theorem states that a square matrix satises its characteristic equation: if p() =

n
+a
1

n1
+ +a
n
, then A
n
+ +a
n
I
n
= 0. Illustrate this for
_
3 3
2 4
_
. Use this to nd the inverse of A.
4.2 Linear Independence and Dependence
Def The nite set of vectors {v
1
, v
2
, . . . , v
n
} is said to be
1. linearly dependent if there exist scalars c
1
, c
2
, . . . , c
n
, not all zero, such that

c
i
v
i
= 0.
2. linearly independent if the only values of the scalars c
1
, c
2
, . . . , c
n
, for which

c
i
v
i
= 0, are c
1
= c
2
= =
c
n
= 0.
The denition can be modied to apply to matrices (n m) instead of vectors.
Ex v
1
=
_
1 2 1

, v
2
=
_
2 1 1

, v
3
=
_
8 1 1

: LD
Ex A
1
=
_
1 1
2 0
_
, A
2
=
_
2 1
0 3
_
, A
3
=
_
1 1
2 1
_
: LI
Given vectors {v
1
, v
2
, . . . , v
k
}, a vector of the form a
1
v
1
+ a
k
v
k
is called a linear combination of v
1
, v
2
, . . . , v
k
.
Thm Let S = {v
1
, . . . , v
n
} be a set of vectors in R
n
. Let P be the matrix whose columns (or rows) are the vectors
of S. Then S is linearly independent i det (P) = 0 i P is nonsingular.
Ex Consider the R
3
vectors earlier. Also, consider S = {
_
1 2 3

,
_
0 1 2

,
_
3 0 1

}
A set of vectors is linearly dependent i one of them is a linear combination of the others. Any two vectors are
linearly dependent i they are proportional.
Thm A matrix square matrix M is nonsingular i its columns are L.I. i its rows are L.I.
Ex Is M =
_

_
1 1 0 2
1 0 1 1
0 1 1 1
3 4 0 7
_

_
. Observe: c
4
= c
1
+c
2
.
MTAP - Introduction to Linear Algebra 13 2nd Semester 2013-2014 (R. Eden)
Ex For what values of c are
_
_
1 0 1
2 1 2
1 1 c
_
_
L.D.?
Ex Suppose S = {v
1
, v
2
, v
3
} is a L.I. set. Is T = {w
1
, w
2
, w
3
}, where w
1
= v
1
+ v
2
, w
2
= v
1
+ v
3
, w
3
= v
2
+ v
3
,
L.I. also?
4.3 Diagonalization and Similar Matrices
Matrices A and D are similar if D = P
1
AP for some nonsingular matrix P. Consequently, A = PDP
1
and
A
n
= PD
n
P
1
. Similarity is an equivalence relationship.
Thm Let A and B be similar square matrices.
1. They are both nonsingular or singular,
2. A
k
and B
k
are similar for any nonnegative integer k (A
k
= I
n
if k = 0).
3. If A is nonsingular, then A
1
and B
1
are similar. Thus, A
k
and B
k
are similar for any integer k. (A
k
=
(A
1
)
k
= (A
k
)
1
)
4. rank A = rank B (skip proof)
5. det (A) = det (B)
6. Tr A = Tr B; recall that Tr MN = Tr NM
7. They have the same characteristic polynomial.
8. They have the same set of eigenvalues (counting multiplicity).
If D is diagonal, so is D
n
, and the diagonal entries are d
n
ii
.
Def A square matrix A is said to be diagonalizable if it is similar to a diagonal matrix: if there is a nonsingular
matrix P such that D = P
1
AP is diagonal.
Ex A =
_
_
5 12 6
3 10 6
3 12 8
_
_
, P =
_
_
1 2 4
1 0 1
1 1 0
_
_
, P
1
=
_
_
1 4 2
1 4 3
1 3 2
_
_
. Then P
1
AP = diag{1, 2, 2}.
Compare with the eigenvalues and eigenvectors of A.
Thm An nn matrix A is diagonalizable i A has n L.I. eigenvectors {v
1
, v
2
, . . . , v
n
}. If P =
_
v
1
v
2
v
n

,
then P
1
AP = diag
1
,
2
, . . . ,
n
, where
i
is the eigenvalue for v
i
.
Ex Can A =
_
1 1
0 1
_
be diagonalized? No since it is defective.
Ex A =
_
1 1
2 4
_
,
1
= 2, (1, 1),
2
= 3, (1, 2). P =
_
1 1
1 2
_
and P
1
=
_
2 1
1 1
_
. Q =
_
1 1
2 1
_
,
Q
1
=
_
1 1
2 1
_
If all n eigenvalues of A are distinct, it can be shown that A is diagonalizable. If however, an eigenvalue has
multiplicity greater than 1, A may or may not be diagonalizable. If for each eigenvalue , we can nd as many
L.I. eigenvectors as its multiplicity, then A is diagonalizable.
MTAP - Introduction to Linear Algebra 14 2nd Semester 2013-2014 (R. Eden)