Chapter 4 Matrices and Vectors

Content
1. Objectives
2. Matrices and Vectors
2.1 Definition of Matrices
2.2 Notation of Matrices
+Standard notation
Column vector notation of a matrix

2.3 Vecors and Scalars

Definition of row matrix and column matrix as vectors
Position vector in
Length or 2-norm of a vector
+Unit vectors and normalized vectors
Direction of a vector in
of a vector and number of elements
Arithmetic operations with vectors
-Addition/subtraction
-Elementwise multiplication
-Scalar multiplication
Dot product and length of vectors
-Important comment on the dot product
Distance between vectors
Orthogonal vectors
Cauchy-Schwartz Inequality
Vector projections and distance
8Example 2.1 Length, direction angle, orthogonality and projection of vectors
3. Working with Matrices and Vectors
3.1 Arithmetic Operations with Matrices
Addition and subtraction
Elementwise opererations
Scalar multiplication
Matrix multiplication
-Definition of matrix multiplication
-Visualization of matrix multiplication using a Falk-scheme
-Properties of matrix multiplication
8Example 3.2 Straight forward multiplication of two matrices

8Example 3.3 Multiplication of two matrices using a Falk-scheme

8Example 3.4 Checking matrix multiplication properties
3.2 Matrix Transpose
Definition
Properties
Checking matrix transpose properties

3.3 Special Matrices

Symmetric matrix
+Zero matrix
+Identity matrix
+Upper/Lower triangular matrix
+Orthogonal matrix
+Submatrix
3.4 Representation of Linear Systems with Vectors and Matrices
Row format representation
Column format representation
Matrix-vector format representation
Linear combination of vectors
+Independent vectors
+Number of independent column vectors an matrix rank
3.5 Solution of Linear Systems Introduction
+Scalar equation
+Matrix-vector equation
3.6 Matrix Inverse Introduction
Definition of the inverse of a square matrix
Usage of the inverse matrix
8Example 3.8 Checking the inverse of a (2 2) matrix
8Example 3.9 Checking the inverse of a ( 3 3) matrix
8Example 3.10 Solving a linear (3 3) system of equations using matrix inversion
Existence of the matrix inverse: singular matrix

4. Determinants
4.1 Introduction
4.2 Definition of determinant using Cramer's rule
system using Cramer's rule
+Unique solution of nonhomogeneous system A x = b for nnonsingular A matrix
4.3

+Definition
Minors
Cofactors
+Adjoint Matrix

4.4
Chess-board sign scheme
Cofactors
4.5 Properties of Determinants
properties
4.6 Easy computation of determinants by sweeping columns

5. Matrix Inversion
5.1 Introduction
Identity matrix
Matrix of cofactors
Chess-board pattern
Adjoint matrix
5.2 Definition matrix inverse using the adjoint formula
Definition matrix inverse
Nonsingular or invertable matrix
5.3
Trick to compute inverse of (2 2) matrix
(2 2) matrix
5.4 Inverse of (3 3) matrices
Matrix inverse using the adjoint formula
Inverse of a (3 3) matrix using the adjoint formula
sing elementary row operations
Inverse of a (3 3) matrix using elementary row operations

6. Solving Square Linear Systems of Equations

6.1 Introduction
6.2 Solving Square n-by-n Systems
1. Matrix Inversion
8Example 6.1 Solving a linear (2

2) system using matrix inversion

2. Direct Elimination
3. Gaussian Elimination and Back Substitution
4. Gauss-Jordan Reduction
5. Cramer's Rule
6.3 Geometrical Solution of Square (2

2) Systems
(2 2) systems graphically

6.4 Solvability of Square Non-Homogeneous (n

6.5 Square (3

n) Systems

3) Systems with no Solutions or infinitly Many Solutions

6.6 Linear Homogeneous Square Systems of (n n) Equations

8Example 6.9 Homogeneous Square Linear System 1
8Example 6.10 Homogeneous Square Linear System 2
6. Problems
7. Solutions

Reference
[1]. Introduction to Linear Algebra, Gilbert Strang , Brooks/Cole 2006, Chapter 1, 2
[2]. Advanced Engineering Mathematics, T. Harman e.a., Brooks/Cole 2000, Chapter 2, 3
[3]. Linear Algebra With Applications, Steven J. Leon, Prentice Hall 2006, Chapter1, 2, 4.
[4]. Feedback Control Dynamic Systems, F. Franklin, Pearson Prentice Hall 2009, Appendix C.

1. Objectives
After completion of this chapter, the reader should be able to
1. Define and recognize a matrix and determine its dimensions.
2. Define a row matrix or row vector and a column matrix or column vector.
3. Know the difference between a scalar and a vector.
4. Compute the length (2-norm) and direction angle of a vector. Normalize a vector to unity
length.
5. State the criteria that must be satisfied for two vectors to be added or subtracted.
6. Compute the sum or difference of two applicable vectors.
7. Know when two vectors are perpendicular.
8. Define a square, symmetric and skew symmetric matrix.
9. Define the transpose of a matrix.
10. State the criteria that must be satisfied for two matrices to be added or subtracted.
11. Compute the sum or difference of two applicable matrices.
12. State the criteria that must be satisfied for two matrices to be multiplied together and determine
the size of the product.
13. Compute the product of two applicable matrices.
14. Determine the determinant
determining the determinant of higher-order square matrices.
15. Define a minor and a cofactor.
16. Define the identity matrix.
17. Define the inverse matrix. Identify a singular matrix.
18. Determine the inverse matrix
determining the determinant of higher-order square matrices.

discuss the algorithm for

19. Formulate a set of simultaneous linear equations in matrix form and solve for the unknown
vector for the case of two equations with two unknowns or three equations with three unknowns
using the following methods: Matrix inverse, Gaussian elimination, Gauss-Jordan reduction, and
Cramer's rule. Solve two equations with two unknowns graphically and show unique solution, no
solution and infinitly many solutions graphically.
20. Know how to perform bases transformations
21. Know how to find out if a square system of linear equations may have infinitly many solutions
or no solutions at all.
22. Find the solution of a square system of homogeneous equations.
23. Compute the eigenvalues and the eigenvectors of a square matrix.

2. Matrices and Vectors

2.1 Definition of Matrices
Matrices provide an orderly way of arranging values to enhance the analysis of systems in a systematic
manner. Their use is very important in simplifying large sets of equations and in determining their
solutions. In particular, computer algorithms for manipulating large sets are simplified greatly by
matrix notation and operations.
A matrix is a rectangular array of constants or variables. For simplicity, we will use constants in
describing the properties. The size of a matrix is described by two integers, which we will designate as
m and n. The first integer is the number of rows ( m ) and the second integer is the number of columns (
n ) of the matrix. We will use upper case italic letters to indicate the name of a matrix in mathematical
equations, such as A, B.

2.2 Notation of Matrices

Standard notation
The standard notation for a general (m
columns is

n) matrix ( read m-by-n matrix) having

rows and

The horizontal lines in matrix A are called rows, the vertical lines columns. The element in the i-th row
and the j-th column is denoted by
that is a lower case italic letter with a double subscript. The first
subscript indicates the row, the second subscript the column. The lower case letter usually corresponds
to the upper case letter used for the matrix, here a and A.
Size or Dimension of a Matrix
Matrix A is sometimes written as
to emphasize on the elements, where the symbol c
denotes "for all". To emphasize that matrix A is of size or dimension (m n) we write
that is

Column matrices and row matrices

Column matrix : is a matrix having only one column.

A column matrix is also called column vector.

Row matrix : is a matrix having only one row

A row matrix is also called row vector.

Column vector notation of a matrix
Column vectors used for this purpose have a lower index corresponding to the column number in the
matrix:

The column vector notation of matrix A then is

where

The column vector notation of a matrix is used in section 3.4 for the column format representation of
a linear system of equations.
Transpose operator
To save (paper) space, column vectors are often written as transposed row vectors, see section 3.2 for
the definition of transposed. Using the transpose opeartor, a column vector with three elements

is then written as the row vector

The exponent T is called the transpose operator.
Similarly a transposed row vector turns into a column vector.
Equiality of matrices
Two matrices A and B can only be equal if all the elements in the first matrix A are the same as the
corresponding elements in the second matrix B . This can be seen from the definition of matrix.
Square matrices and trace
If the number of rows is equal to the number of columns, that is if
the matrix is said to be
square. In this case the notation A(n n) could be used to denote the matrix if the size is important.
Standard notation for a general square matrix

The elements
form the main diagonal of the square matrix. The other elements are called
the off-diagonal elements.The sum of the main diagonal elements is called the trace of square matrix,
hence

Overview
The notation A =
The rows of matrix

, means that matrix A has the elements

and

are numbered from top to bottom with

The columns of matrix

are numbered from left to right with

(m n) is called the size or dimension of the matrix, notation

Two equal matrices : A =

and B =

are equal if

m n) and
=

The trace of a square matrix is the sum of the main diagonal elements.

2.3 Vectors and Scalars

,c

= (m n).

Definition of row matrix and column matrix as vectors

Remember, a matrix having only one row is called a row matrix. A matrix having only one column is
called a column matrix. In many applications, a matrix of either form is called a vector. We will use
lower case boldface letters for the vector names, thus a typical row or column vector might be
indicated as x or as y.
scalar and is the form of most variables considered in simple algebraic
equations. We will use greek letters or italic lower case letters for scalars such as
Vectors are often used to represent solutions to linear systems. For example, the solution
linear system of equations

can conveniently be represented by the column vector

row vector x =
that, in contrast, the notation

, but also by the transposed

. The symbols and are the scalar elements of the vector. Notice
represent the cartesian coordinates of the point x in a plane

In working with linear systems, it is generally most convenient to represent the solutions in terms of
column vectors or (n
n
matrices of real numbers is called Euclidean
n-space and is usually denoted by
Thus
and are our familiar spaces "line", "plane" and "space". The corresponding column
vectors are called 1-D, 2-D or 3-D vectors, D for dimensional.
Position vector in
We represent (position) vector
in as an arrow from origin O(0, 0) to point P(
) in a
cartesian coordinate system, as in Fig. 1, using standard orthonormal basis vectors
, such that
usually written as
To show that vector a now is a general vector, so
not a specific row or column vector, we use a notation with comma's between the elements. Remember,
row and transposed column vectors are written with spaces between the elements.

y-axis
a , P(a1, a2)

a2
a
e2
O (0,0)

q
e1

a1

Fig. 1a Position vector in =2

x-axis

Length or 2-norm of a vector

Given the vector a =
in

see Fig. 1. This is a two dimensional vector, shortly 2-D vector.

The length of vector a is the length of the line segment OP and can be computed with Pythagoras as:

and is called Euclidean norm or 2-norm, to indicate that the vector lengths are squared. The length of
a vector is also named magnitude, specially when the vector represents a physical quantity (velocity,
force etc). Similarly, the length of a 3-D vector a =
is

Generalization to n dimensions is straightforward:

Unit vectors and normalized vectors

A vector a of length 1, that is
is called a unit vector. For any vector a, a unit vector u in the
direction of a can be computed as

assuming that
is not zero. This unit vector defines the direction of vector a. A nonzero vector
divided by its magnitude is said to be normalized, and has length 1.
In the rest of this chapter we will omit the subscript 2 in
is the only norm we use!

to indicate that it is the 2-norm, since this

Direction of a vector in
The angle between the positive x-axis and the vector a (taken in anti clockwise direction) determines
the direction (or orientation) of the vector. It is often called the direction angle of the vector, and is
usually measured in radians.
The direction angle for vector a lying in the first and fourth quadrant can be computed with

For vectors in the second quadrant we must add radians while for vectors in the third we must
subtract
Table 2.2 in Chapter 1: Complex Numbers. To be sure, always draw the

vector in

Any two arrows having the same length and the same direction are said to represent the same vector.
Only arrows that start at the origin are called position vectors; all other vectors are so-called free
vectors.
Size of a vector and number of elements
The size of a vector is equal to size of the corresponding column matrix or row matrix. For example,
the size of vector
considered to be a row vector

However, the size of vector

considered to be a column vector
that is
The number of elements is in both cases of course 3, so
Arithmetic operations with vectors

Addition/subtraction

Elementwise multiplication

All operations are done element wise, hence the vectors hust have the same dimension.

Scalar multiplication

A negative value for scalar

used in the scalar multiplication

reverses the direction of vector a

A positive value for scalar

stretches it

less than 1 shrinks the vector, while a positive value greater that 1

In Matlab, except element wise multiplication with

, elementwise division with
and
elementwise exponentiation with ^ b are also possible. The elementwise operator, borrowed from
Matlab, consists of a dot, immediately followed by the symbol of the operation that must be
performed elementwise. See also Matlab solution of Example 2.1.
Dot product and length of vectors
The vector dot product of a and b, written as
(1)

can be defined in two ways as:

, or sometimes as

so with a dot between the vectors.

(2)

where is the smallest angle between a and b.

Number (1) is just a definition and (2) can be proven using the law of cosines in a triangle formed with
the vectors a, b and
as sides.
Important comment on the dot product
1. The vector dot product is a scalar ( a number) and is therefore called scalar product. It is also
called inner product.
2. The order of a and b in the dot product makes no difference, so
3. Choosing

in the first definition of the dot product, we have

4. The length (or 2-norm) of vector a can thus also be determined using the dot product:

5. Knowing the dot product and the vectors we can compute the angle
for

and the smallest angle between the vectors.

6. The dot product (inner product) can be computed as the product of the vector and its transpose as
follows:
If a and b are column vectors of equal length n , so
transpose operator T,
the inner product
can be computed with
Falk schemes for

Observe that both multiplications delver the same as

and

then using the

, as easily can be shown by using the following two

, however using the transpose is better since it

is also applicable for matrices. Go to Example 3.6. to see an application.

Distance between vectors

The distance between two vectors a and b is defined as the number
and

then

so

Orthogonal vectors
1. If the dot product

2. In

and

For example if
Draw the vectors to check.

is zero then the vectors are perpendicular (or orthogonal) if

, the vectors thus meet at right angles, since cos

hence

a b 5
3. If orthogonal vectors are of unity length they are called orthonormal.
Cauchy-Schwartz Inequality
If a and b are two vectors in either in

or

then the Cauchy-Schwartz inequality says

The equal sign holds only if one of the vectors is the zero vector 0 or if the vectors are colinear, that is
if they are lying on the same carrier.
The inequality follows from the fact that
for vectors that are not pointing in the same or
opposite direction.
Vector projections and distance
The inner product can be used to find the projection vector p of a vector b on another vector a, see Fig.
1b.

y-axis
6

L
d = b Kp

y = 0.5 x

4
a

5.6
p
q

x-axis
4
O (0,0)

2.8

Fig. 1b Vector projection

The projection of vector b on vector a is given by the vector

Note that vector p is lying on line L in the direction of vector a, while vector
perpendicular to p.

is

Derivation
For the length of projection vector p we find
= ||b||
The dot product of a and b is

hence, we can express the length of p with the dot product as follows

=
Vector p has the same direction as vector a, so we can write
,

So, using the length of vector p we find for the scalar multiplier

we find

For example if

then

so

The shortest distance of the tip of vector b to the carrier of vector a, that is to the line
the origin is

and

through

Example 2.1 Length, direction angle, orthogonality and projection of vectors

(a). Compute the length and the direction angle in degrees of vector a =
(b). Check if the vectors a =

and b =

are orthogonal.

(c). Find the angle in degrees between the vectors a =

and b =

(d). Find the projection vector p of vector b on vector a and the shortest distance of b to the line
through a.

Solution
(a) Length and the direction angle of vector a =

y-axis
a = K3, 4

5
4

a =5

q = 127 +

0,

K4 K3

x-axis

(b) Orthogonality of vectors

a=

and b =

The dot product is zero while the length of each vector obviously is not zero 0 a

z-axis

y-axis
q = 90 +

b = 6, K2, 1

a = 2, 5, K2
x-axis
(c) Angle between vectors
a=

and b =

=
(smallest angle between a and b )

z-axis

q = 97 +

y-axis

b = 3, K2, 1
a = 1, 2, 0
x-axis

(d). Projection vector p of vector b on vector a and the shortest distance of b to the line through a.

Difficult to check geometrically!

Software help

3. Working with Matrices and Vectors

The basic arithmetic operations with vectors were already covered in section 2.3. Some of the basic
arithmetic operations with matrices will be discussed in this section. In some ways, they follow the
regular operations associated with scalars and vectors, but there are some peculiarities that must be
carefully observed.

3.1 Arithmetic Operations with Matrices

Addition and subtraction
Matrices can be added or subtracted only if they are of the same size. Let
two matrices of the same size, then

represent

where

Elementwise operations
These operations can also only be done if the matrices have the sane size, then the elementwise
multiplication is defined by

where
v
Elementwise division and exponentiation is done with the * replaced by / respectively ^, see section 2.3,
where the same is done for vectors.
Thus, for addition, subtraction and elementwise operations, corresponding elements are added ,
subtracted respectively elementwise operated on. The resulting matrix is of the same size as either of
the matrices being combined, thus

Scalar multiplication
If A is a matrix and is a scalar , then is the matrix formed by multiplying each of the elements of A by
thus
A=

The multiplication dot is often omitted when no confusion is possible. Arithmatic operations and
elementwise operations will be illustrated in Example 3.1

Example 3.1 Arithmetic operations and elementwise operations with two matrices

Two matrices are given by

where and are scalar variables. Determine (if possible) for

the

(a). Sum of two matrices

(b). Difference of two matrices
(c). Scalar multiplication of a matrix
(d). Elementwise multiplication of two matrices

Solution

(a).
.The solution is possible because both matrices have the same size. We simply add the
corresponding elements and obtain:

Don't be disturbed by the + sign in the third column. Matrix C has a combination of variables and
constants, so the result will have the same property. If neccessary you could have used some
parenthesis:
(b)
have

If

(c)

. In this case, the elements of B are subtracted from the corresponding elemets of A. We

then

. Here each element of D is multiplied by 2, so

(d)

. Element wise multiplication of matrices A and D yields

Software help
Matrix multiplication
Definition of matrix multiplication
Multiplication of two matrices is a bit tricky and requires a little practice. In order for it to be possible,
the number of columns of the first matrix must be equal to the number of rows of the second matrix.
This means that the matrix multiplication is not commutative except in very special cases. In general

Thus, the order of the multiplication is very important. Here, the matrix multiplication operator is again
denoted by a small dot as in A. B , to distinguish it from the scalar multiplication operator denoted by a
somewhat higher dot as in
. Both operators however are often left away when there is no confusion
possible.
Let C represent the product of the two matices A(m n) and B(n
p) , thus we have
A(m n) B(n

p) , the size of the product is then (m

p) = C(m p)

Any element in the matrix C can be obtained from the elements of matrices A and B with the
following algorithm:
and
To see what this really means we will visualize the process.
Visualization of matrix multiplication using a Falk-scheme
To form
, the number of columns of A must equal to the number of rows of
called compatible matrices.
To find the

entry of the product

If so, A and B are

you need the i-th row of A and the j-th column of B.

Form the product of the corresponding entries and add up:

is the dot product of the two vectors found at row i of matrix A and column j of matrix B.

In fact

means that the i-th row of A is orthogonal to the j-th column of B.

Remember, the matrix product is not commutative, that is
The matrix multiplication process can be visualized with the so-called Falk-scheme (Fig.2) which
will be used in several examples that follow.
j

1j

2j

nj

i1

i2

in

ij

Fig. 2 Falk Scheme for Multiplying two Matrices

Properties of matrix multiplication

If A and B are n
general

n square matrices, the products

does not imply

or

where

and

are definitly defined , but still in

is the zero-matrix , a matrix with all zero

elements.

Example 3.2 Straight forward multiplication of two matrices

Given the following matrices:

Find

and

if possible using straight forward matrix multiplication.

Solution
A(2

The product

3) B(3 2) = C(2

2).

B(3

The product

2) . A(2 3) = C(3

3).

Comment
Notice that
Handcalculation of matrix multiplication is quite involving and cumbersome.
However using the Falk-schema makes live easier, see Example 3.3.
If possible, always use some reliable software to check your hand calculations like Matlab.
Software help

Example 3.3 Multiplication of two matrices using a Falk-scheme

Given the matrices

Find

and

if possible, using the Falk-scheme for matrix multiplication.

Solution
(a). The required Falk-scheme is:
B

To determine
we multiply the first row of the first matrix (A) by the first column of the second
matrix (B) on a element-by-element basis, and then add the results using the scheme above. This
yields

To determine
giving

we multiply the first row of A by the second column of B in a similar manner,

To determine

we multiply the second row of A by the first column of B , giving

Finally, to determine

we multiply the second row of A by the second column of B , giving

The product matrix C is then given by

Notice that in determining the various coefficients, there are always an equal number of elements in the
row of the first matrix being multiplied with the number of elements in the columns of the second

matrix. This is a result of the fact that the number of columns of the first matrix must be equal to he
number of rows of the second matrix.
(b). The required Falk-scheme now is
B

In this case, we need to determine nine different constants, each of which is determined by multiplying
the two elements of a row of B by the two elements of a column of A and adding the products. Using the
scheme below we can easily compute the product.
B

To check your computation you always can use appropriate free software like Wolfram Alpha. Try It!
Software help

Example 3.4 Checking matrix multiplication properties

Given the matrices

and

Find
(a).

and

and show that

(b).

and show that

while C and D

( O is the square zero matrix here).

Solution
(a).

A B=

B A=

Obviously

so matrix multiplication is not commutative!

(b).
Obviously

does not imply

where O is the null matrix.

Software help

3.2 Matrix Transpose

Definition
The transpose of a matrix A, denoted by , is obtained from A by interchanging the rows and
columns. Thus, if A has size (m n) , then will have size (n m). Note that the transpose of a
row vector is a column vector and vice versa. The exponent T in
is called the transpose operator.
As already used in section 2.2, large column vectors are often written as transposed row vectors to save
(paper) space:

Properties
If

is a scalar and

and

are (m

n) matrices then

If a and b are column vectors of length n , so

Using the transpose operator, the inner product
column vectors, see Example 3.6.
If a and b are row vectors, the inner product

and

then

can be computed with

can be computed with

Each property can easily be proven by using the definition.

Example 3.5 Checking matrix transpose property

Given the matrices

Show for these matrices that

Solution

First A B using a Falk scheme

Then

using a Falk scheme, must be equal to

, if a and b are

Software help
Two column vectors a and b in can be consideres as (n
can the form the matrix product
or,
more simply, as a real number . The position of the transpose operator and also order of the vectors is
important. Look at the solution of next example to see that
and
do not deliver the dot product
but different 3-by-3 matrices. Do it!

Example 3.6 Matrix-vector products

Given matrices

Find
(a). What kind of vectors/matrices are a, b and C?
(b). Determine the inner product
(c). Determine the products,

. Comment on the type of the result.

Comment on the type of the result.

Solution
(a)

3-D row vector , can also be considered as a

3-D column vector, can also be considered as a

3-D column vector b , can also be considered as a

matrix, call it

matrix, call it A.

1) matrix, call it B .

C.

(b)

Determine the inner product

. Comment on the type of the result.

Comment The result is clearly a scalar (number).

The inner product also can be computed using transposed vectors:

Falk schemes

Using the wrong order of the vector and transposed vector, as in

3-by-3 matrices. Using alk schemes we have

(c). Determine the products,

(c1) Vector b can be considered as matrix

and

, results in two differennt

Comment on the type of the result.

, hence

=F

Comment The result F is a 3-by-1 matrix or column vector as result of a matrixvector product.

(c2)

Vector

can be considered as matrix

, hence

Comment The result G is a 1-by-3 matrix or row vector as result of a vector-matrix product.

Software help

Example 3.7 Size, trace, transpose and product of matrices

Given the matrices

Find if possible
(a). The size , trace and the transpose of A, B and C.
(b).

Comment on the result.

Solution
(a) The size , trace and the transpose of A, B and C.

trace

is only defined for square matrices:

transposes

(b)
Falk-scheme for

and

Comment on the result.

3)

C(

Falk-scheme for

2)

D(

Comment

are both square matrices , but different values and different sizes.

Software help

3.3 Special Matrices

Symmetric matrix
A square matrix A is called symmetric if
For example

for every element, or equivalently if

If
then the matrix is called skew-symmetric. One type of application that leads to symmetric
matrices are problems involving electric networks and skeleton structures. Symmetric matrices have
nice properties, for example they always have real eigenvalues and orthogonal eigenvectors, and
therefore are always diagonalizable.
Some matrices have special structures and properties and are given dedicated names.
Diagonal matrix D : A diagonal matrix D is defined as a symmetric matrix in which all the offdiagonal elements are zero.
Zero matrix O : The square matrix for which all elements are zero is called a zero matrix O.
Identity matrix I : The n
matrix I.

n diagonal matrix such that

is called the identity

The square zero matrix and the identity matrix play the same role in matrix algebra as the "0" and the
"1" in normal algebra.
Upper/Lower triangular matrix U and L : A matrix is said to be upper triangular if all elements
below the main diagonal are zeros.

A variant is the lower triangular matrix

where all elements above the main diagonal are zero.
As we will see in section 5, the determinant of a diagonal or triangular matrix is simply the product of
its diagonal elements.
Orthogonal matrix Q : A square matrix is called orthogonal when the column vectors are
orthogonal in pairs. In
the column vectors are then perpendicular. Orthonormal matrices
are orthogonal matrices with column vectors of length 1. As we will see later, a square matrix Q is

orthogonal when
Submatrix A submatrix of A( m n) is the matrix A with certain rows and columns removed.
Block matrix A block matrix or partitioned matrix is a matrix which is broken into submatrices.
Matlab has a group of functions to generate special matrices. We will introduce these functions on need
to know basis.

3.4 Representation of Linear Systems with Vectors and Matrices

Row format representation
Given the following system of two linear equations in two unknowns, shortly called linear system:

This system is given in the so-called row format form: each row is precisely one equation. We can
easily solve this system algebraicly by hand using elimination. Multiplying the second equation by 2
and adding it to the first delivers

so ,

and

If we graph the two straight lines the intersection point is indeed the solution:
This point satifies each of the two equations.

x
row 2
x = 0.5 x C1.5
2

K3

1.5

x = 1, x = 2
1

K1

K2

K1
row 1
x = 0.5 $x
1

Fig. 3 Straight lines of the row format form

Column format representation

An alternative way to represent the linear system of equations is to express it as a sum of column
vectors:

which is called the column format form. Using this alternative we can represent the linear system as a
vector-matrix equation of the form:

where

is the first column and

is the second column vector of matrix A, so

see also section 2.2 for the column vector notations of a matrix. Notice that the vectors
are lower case boldface letters with a bold subscript, thus indeed written as vectors.
In fact the right-hand side vector b is a linear combination (see below for the definition) of the column
vectors of matrix A. We already know that
so the linear combination takes the form

which is perfectly correct. We now draw a picture of the geometry in

x2

0
b=

3
2

K1

2$

K1
2
1K

a2
K1

x1

a1
K1 K

2
1$

K1

Fig 4. Vectors of the column format form

Observe how the rhs vector b

is composed of one time
plus two times
, thus a linear combination of the two column vectors of matrix A. When we take all
possible combinations of the two column vectors we get the complete
plane.
Matrix-vector format representation
We can represent the row format and the column format together very compactly using the matrixvector product as follows

where A is the system matrix consisting of the coefficients of the three equations, is the vector of
unknowns and b is the vector of right-hand-side constants. Using this method we can thus represent
the linear system as a matrix-vector equation of the form

Observe that in this particular case the sizes of the A and x match to produce b of the form
A(2

2) . x

1) = b

Notice that we can read the two equations back from the vector-matrix form by simply performing
matrix-vector multiplication. The column format however can best be seen from the linear system's

equations.
Linear combination of vectors
The linear combination of a set of vectors

is defined as follows

If
are vectors in and
linear combination of these vectors if

are scalars , then vector y is called a

y=
that is, if y can be written as a sum oof scalar multiples of these vectors

Therefore the matrix-vector product A x is a linear combination of the columns of A. Thus, if A( n

and x is a vector in
then

n) =

is a linear combination of the column vectors of matrix A. To show this is right consider the following
2-by-2 case.

For a 2-by-2 matrix A( 2

)=

the matrix-vector representation of a set of

two linear equations is

or, using the column format representation,we have

so it must be true that

is a linear combination the column vectors of A , so

Independent vectors
The linear independency of a set of vectors

If

are vectors in

and

is defined as follows

are scalars , then these set of vectors is

called linearly independent if

is true only for

Otherwise these vectors are linerly dependent and at
least one of them is a linear combination of the others.
Two vectors are independent when they are not lying along the same carier. The two column vectors
and
of matrix matrix-vector equation used above, repeated here for
convenience

are thus independent, because they are separately lying in

see Fig. 4. The definition for the linearly
independency of a set of vectors says that it is impossible to express one of the vectors as a linear
combination of the others.
Hence, the two vectors a and

are linearly independent if

is true only for

where the
are scalars. It is easy to verify that this is indeed the only
possibility, so the two vectors must be independent.
Number of independent column vectors and matrix rank
It can be shown, see section 6, that the columnvectors of a square matrix A are independent if its
determinant is unequal to zero, so if
The number of independent column vectors of a matrix A is given by the so-called rank of matrix A ,
notation
. If all column vectors of matrix
are independent, we say that the matrix has
full rank and
Matrix rank is used in section 6 for determining the solvability of a linear
system of equations.

3.5 Solution of Linear Systems Introduction

Scalar equation
In the preceding sections we have defined and used the most important matrix operation: the
multiplication of two matrices. Much of the motivation behind it comes from the application to linear
systems of equations. If we have a system of one linear equation in one unknown, it can be written as

The solution to this scalar equation can be written as

where

We generally think of

and b as being scalars. However, they could also be treated

Matrix-vector equation
Our goal now is to generalize the scalar equation to the matrix-vector equation

so that we can represent an m n linear system by a single matrix-vector equation , where A is an m

n matrix, x is the vector of unknowns in
and b is a vector of known constants in
. Here we will
consider the case where
, so A is a square matrix which is the normal case. The cases
and
will be considered fully in the Math 1 course.

3.6 Matrix Inverse Introduction

Inverse matrix concept
The solution to the matrix-vector equation will have the same appearance as the solution to the scalar
equation:

However here
is not equal to 1/A
further elaborate the concept of inverse matrix.

In the next section we will

Definition of the inverse of a square matrix

There is no such thing as division in matrix algebra. The process of matrix inversion is the closest thing
to division. The inverse of a square matrix A is defined as the matrix
so that

or, more general if A is an n

abbreviated as iff)

where I is the (n
normal algebra.

n matrix, then matrix B (n

n) is the inverse matrix of A if and only if (

n ) identity matrix. The identity matrix plays the same role as the number 1 in

Usage of the inverse matrix

The next three examples show the usage of the inverse matrix.

Example 3.8 Checking

Given the matrices
Check that matrix B is the inverse of matrix A and vice versa.

Solution
Using a Falk-scheme we have

Thus
the inverse of square matrix A. Check
identity matrix.

by your self and see that it is also equal to the

Software help

Example 3.9 Checking the inverse of a

matrix

Given the matrices

Check that matrix B is the inverse of matrix A.

Solution
Using a Falk-scheme we have

________________________

Hence

Check by your self that indeed

Software help

Example 3.10 Solving a linear

) system of equations using matrix inversion

Given the following system of three linear equations in three unknowns in row-format:

Find
(a). The linear system written in column format. Identify the various scalars, vectors and matrices if
present.
(b). The linear system written in matrix-vector representation. Identify the various scalars, vectors
and matrices if present.
(c). The solution of the system using the matrix inverse of Example 3.9

Solution
(a) Column format

where

is the i-th column vector of matrix A given in part (b).

(b) Matrix-vector representation

where A is the system matrix consisting of the coefficients of the three equations, is the vector of
unknowns and b is the rhs vector of constants.
(c) Solution of the system

see Example 3.9.

Hence, the solution is

substituting the solution into the equations.

Software help

which easily can be checked by direct elimination or by

Existence of the matrix inverse : singular matrix

Not all n n matrices have an inverse matrix that is are invertable. A matrix which is not invertable is
called singular or not invertible. An invertible matrix is called nonsingular or regular. The matrix
inverse is unique, meaning that each square matrix has only one inverse. The process of finding the
inverse of a matrix will be considered in detail in a later section, after we have considered
determinants in section 4.

4. Determinants
In this section we use determinants to solve linear systems. First, we use Cramer's rule for solving a 2
on the fly. We also will need
determinants for matrix inversion. So let's start quickly!

4.1 Introduction
The determinant of a matrix can be determined only for a square matrix. Unlike a matrix, a determinant
has a single value. There are two different notations for a determinant. For a matrix A, the determinant
can be represented as
or simply as
There are many "tricks" that appear in mathematics texts to simplify the process. The problem with
some of these tricks is that they work fine when the values in the matrix are "clean" numbers, but the
process becomes more burdensome when the numbers represent "real" data.
With the ready access to such mathematical packages as Maple, Matlab and the free software like
Octave , FreeMat and Scilab but also Wolfram Alpha in the scientific community, there is little reason
or justification for trying to accomplish the chore by hand for anything but the simple case. For that
reason, we will limit our development here to the simple
development of the latter, we will introduce the concept of expansion of minors, which in theory could
be used for any determinant.
The simplest matrix is a trivial 1 x 1 matrix. This is really a scalar, and its determinant is simply its
value. Therefore, we need not consider it any further.

4.2 Definition of determinant using Cramers's rule

Usage of determinants simplifies the solution of linear systems of equations. As an example consider
the following linear system of two equations with two unknowns:

Eliminate

by multiplying the first equation by

Now, solving for

we have

and the second by by

and subtract

Now eliminate

Solving for

by multiplying the first equation by

and the second by

and subtract

now delivers

Now, looking at the vector-matrix equation, repeated here for convenience

we can define as a kind of short hand notation:

That is, the deteminant

the product of the other two elements.

A is the product of the main diagonal elements minus

As a consequence, we also can use the following abbreviations:

Now, the solutions can be very handy and shortly written as

Now, we call by definition

the determinant of matrix A. The solution method used is called Cramer's rule. This rule can be
generalized to more then two dimensions, see section xx.

Example 4.1
Solve the following system of two linear equations using Cramer's rule.

Check the solution by substituting the solution back into the equations.

Solution

Notice the extensive use of paraenthesis in computing the various determinants to avoid sign errors!

Check

Solftware help
Unique solution of nonhomogeneous system
If the system of equations is nonhomogeneous (
only if matrix A is nonsingular, that is if

for nonsingular A matrix

) , it has a unique solution

If the system of equations is homogeneous (

solution

) and

it has only the trivial

If matrix A is singular, that is

the homogeneous system has also nontrivial solutions , see
later when we come to eigenvalues in section x.x.

more systematic way.

4.3 Determinant of (

) matrices

Definition

is defined as:

the product of the other two elements. A common error in evaluating determinants is that of keeping the
signs straight, since some of the elements may have negative signs. See also the Hand Calculation
solution of Example 4.1 and Example 4.2.

Example 4.2
Determine the determinant of the matrix

Solution
The determinant is

Notice the cancelation of the minus sign in the second term in this example. It is therefore useful to use
parenthesis around all elements.
Software help
Minors

minor. Let
represent a minor
corresponding to the i-th row and the j-th column of a matrix A. If A is a square matrix of size n n ,
is a determinant of size
by
formed from the matrix by crossing out the i-th row
and the j-th column and evaluating the resulting determinant.
Cofactors
A cofactor will be denoted as

It is equal to the corresponding minor

by the relationship

Thus, cofactors are alternaterly equal to the minors or the negatives of the minors. Using a chessboard sign scheme, see section 4.4 and Example 4.4 below, the sign of cofactors can easily be
determined.
The significance of minors and cofactors is that a higher-order determinant can be expanded into the
sum of lower-order determinants. For example a
determinants.We easily can see how the number of smaller determinants can quickly grow as
successive expansion is performed.
Adjoint Matrix
The adjoint matrix is the transpose of the matrix of its cofactors:

The adjoint matrix will be used for calculation the inverse of a square matrix, because it can be shown
that

No proof is given here. If

then multiplication at the left with
of both sides, delivers the
so-called adjoint formula for computing the inverse of a square matrix A :

Matrix inverses will be computed in Section 5.

4.4 Determinants of (

) matrices

Definition
the first row, the determinant is defined as

The individual cofactors can be computed as (notice the red plus and minus signs)

The usage of cofactors and minors will become completely clear in Example 4.3 and Example 4.4
below.
Chess-board sign scheme
Using a chess-board sign scheme, see Example 4.4 below, the sign of cofactors can easily be
determined. Clearly, this is as far as we need to go.

Example 4.3 Mino

matrix

Determine the nine minors for the following matrix

Solution
Although tedious, the procedure is straightforward. Starting with

cross out the first row and the first

row and then repeat the procedure on the second and third rows. The nine minors are calculated as
follows:

Software help

Example 4.4
Determine the cofactors of the nine minors determined in Example 4.3.

Solution
The cofactors are directly related to the minors with only a sign change for alternative values. The
relationship is

All this means is that if (

is an even integer, the cofactor is equal to the minor, and if
is
an odd integer the cofactor is equal to the negative of the minor. Thus, we can write down the cofactors
as:

Using the chess-board sign scheme below, the cofactors can more easily be formed from the minors:
Chess-board

Minors

Cofactors

Software help

Example 4.5
Determine the determinant of matrix A of Example 4.3 by expansion in cofactors along the
(a). first row of matrix A, and

(b). first column of matrix A.

Solution
(a). Determine

by expansion in cofactors along the first row of A.

Matrix A

Cofactors

Determinant of A by expansion along the first row now becomes

We only need three of the nine minors for detemining the deteminant. However, the other six will be
used later when we calculate the inverse matrix. Substitution of the values from Example 5.4 leads to

=
=

(b). Determine

by expansion in cofactors along the first column of A.

Since we do not need all minors we simply expand along the first column keeping track of the correct
signs using the check board sign scheme:

=
Using parenthesis around each element and square brackets to group each expansion as shown

This is as far as we go doing handcalculation for deteminants. For determinants greater than
everybody uses software from internet like for instance Wolfran Alpha. Have a look and enjoy!
Software help

4.5 Properties of Determinants

Below are listed eight important properties of deteminants without proof.
List of eight properties
Determinants are only defined for square matrices.
Given two square ( n
1.

n ) matrices A and B :

if A has a row (or column) of all zeros.

2. If any two rows (or columns) of A are interchanged, the sign of

3. Multiplying a single row (or column) by a scalar

is changed.

results in

4. Adding a multiple of one row (or column) to another does not change the value of the determinant.
5.
6.
7.
8.

(notice the exponent!)

4.6 Easy computation determinants by sweeping columns

Using these properties allows us to quickly compute the deteminant of a matrix. For instance, by
creating zeros in a column using property 4 expanding along such a column is easy done. We give an
example of this so-called sweep process.

Example 4.6
Determine the determinant of the matrix of Example 4.3 by sweeping the first column with the first
row.

Solution
Sweep the first column, creating two zeros:

add R2 to R1

add 3 times R2 to R3 :

stop : 2 zeros in column 1

Now, expand along the first column keeping track of the correct sign:

Software help

Example 4.7 Determinants of diagonal, upper triangular and lower triangular matrix
Determine the determinant of the following matrices: diagonal matrix D, upper triangular matrix U and
lower triangular matrix L :

using the properties of determinants.

Solution
Expanding along the first row (or column) shows that for all matrices, the determinant is simply the
product of the main diagonal elements:

Software help

Example 4.8 Calculating a determinant using various properties

Determine the determinant of the matrix

using the properties of determinants.

Solution
Since the matrix can be written as

Now, expanding

along the first row gives

The determinant of matrix B therefore is

Software help

where

the determinant is

5. Matrix Inversion
5.1 Introduction
We now turn our attention back again to the concept of matrix inversion, which will be one of the last
major matrix manipulations of this chapter. Determination of the inverse of a matrix is one of the most
difficult processes to perform manually but is quite simple to perform with commercial available
software like Maple and Matlab or with the free software alternatives like Octave, FreeMat or Scilab
and also Wolfram Alpha to mention a few.
The inverse of a matrix will be considered here only for the case of a square matrix, which is the
usual application. There is also a concept called the pseudo inverse of a nonsquare matrix, but that is
beyond the scope of the present treatment. Inverses of square matrices will be used to solve systems of
linear equations, see Section 6.
As in the case of determinant calculation, we will limit how far we go with the manual process: we will
only invert 3-by-3 matrices. The matrix to be inverted will be called original matrix. For your
convenience we now repeat some important topics needed for matrix inversion.
Identity matrix
Remember, the identity matrix is to a matrix what the number 1 is to a scalar. It is a square matrix with
the value 1 along the main diagonal and 0 at all other positions. It will be denoted by I and will have the
form

The actual size of the identity matrix I will become clear from the context and can be adjusted for any
particular case as long as the number of rows is equal to the number of columns.
Matrix of cofactors
A matrix of cofactors is a square matrix in which each element of the original matrix is replaced by its
cofactor. It will be denoted by
and can be represented by

A cofactor it self is defined as

where
is the minor found by crossing out the i-th row and the j-th column in the original matrix and
evaluating the determinant.

Chess-board pattern
The plus- and minus signs, according to
, in front of the cofactor definition above, follow a
checkboard pattern with a plus sign in the upper-left corner:

Adjoint matrix
The adjoint matrix is defined as the transpose of the matrix of cofactors. It will be denoted
can be expressed as

and

The adjoint matrix is sometimes called adjugate matrix.

5.2 Definition of matrix inverse using the adjoint formula

Definition matrix inverse
Using the definitions that have been introduced, one way of finding the inverse matrix is by using the
adjoint formula, that is

see also Section 4.3. It is of course assumed, that

that is matrix A is nonsingular.

The adjoint formula looks deceptively simple, but remember that there are quite a few operations
required to obtain the adjoint and the determinant. Note that the determinant in the denominator is a
scalar and all elements of the matrix in the numerator will be divided by the value of this scalar. It is
perfectly clear from the adjoint formula that the inverse of a square matrix only exists if it deteminant is
nonzero.
Nonsingular or invertable matrix
Only a square matrix can have an inverse
If
If

then matrix A is said to be nonsingular, regular or invertible and

then matrix A is said to be singular or non-invertible and

exists.

does not exists..

5.3 Inverse of (

) matrices

using the adjoint formula

Chess-board

signs:

Cofactors:

Matrix of cofactors:

Adjoint matrix:

Determaint:

Inverse:

Trick to compute inverse of

A very useful trick to compute the inverse of a

All you have to do is:

1. switch main diagonal elements,
2. change sign of other two elements, and
3. divide by the determinant ( a number).
We have already encountered this "trick"in section 4.2 when solving a system of two linear equations
with Cramer's rule.

Example 5.1 Inverse of a

Determine the inverse matrix of
and

using Falk schemes.

Solution
Matrix

Determinant

Inverse

Check that

Matrix
Determinant

Inverse

Now, check for yourself that also

Software help

5.4 Inverse of (

) matrices

In this section, we will compute the inverse of a 3-by-3 matrix by using (1) the adjoint formula and (2)
by using elementary row operations.
Matrix Inverse using the Adjoint Formula
Inverse matrix
Determinant
Adjoint matrix
Matrix of cofactors

Before starting always first check if the matrix is invertable with

Example 5.2 Inverse of a

matrix using the Adjoint Formula

Determine the inverse matrix of A of Example 4.3 using the adjoint formula and check the result.

Solution
The matrix is repeated here for convenience.

Most of the computations required have already been made in preceeding examples. The
determinant is already determined in Examples 4.5 and 4.6:

The cofactors were computed in Example 4.3:

The adjoint matrix is determined by taking the transpose of the matrix of cofactors

Finally, the inverse matrix

is determined with

Checking can be accomplished (1) by computing

by the using software.

which is left as an exercise to the reader or (2)

The matrix A above will appear again in Example 6.2 in the next section.
Software help
Matrix Inverse using elementary row operations
Although the adjoint matrix and the determinant can be used to compute the inverse this is however
not generally used in computer solutions. The method using elementary row operations is usually more
efficient for numerical computation and for hand calculation with "easy" numbers.
To find the inverse of a square matrix, we use elementary row operations:
1. Replace a row of the matrix by a nonzero multiple of the row
2. Replace a row by the sum of that row and a multiple of another row
3. Interchange two rows of a matrix
The elementary operations on rows (or on columns, not both) of matrices may be performed because of
the linearity properties of matrices. Elementary row operations are also used for the Gaussian
elimination and the Gauss-Jordan reduction to find the solution of a system of linear equations, see
section 6.2. We now give an example of using elementary operations to find the inverse of a 3-by-3
square matrix.

Example 5.3 Inverse of a (

Find the inverse matrix of

Check the result by checking if

matrix using elementary row opeartions

using elementary row operations only.

the 3-by-3 identity matrix.

Solution
Start with
Now manipulate the columns of A and I simultaneously in such a way that matrix A
becomes I, and consequently I becomes
because

Doing so, start with

Now perform the following elementary row operations simultaneously on both matrices:
(1) Add first row to third row

(2) Multiply third row by

(3) Subtract third row from first row

(4) Multiply first row by

(5) Finally rearrange columns of left matrix to the identity matrix giving

Check result

This process to find the inverse of a square matrix is known as Gauss-Jordan reduction, see also
section 6.2.
Software help

6. Solving Systems of Linear Equations

6.1 Introduction
There are numerous practical applications of matrices, some of which we will encounter later in the
Control System Engineering classes. One of the most basic applications of matrices is in solving
simultaneous linear equations. When we have a set of n linear equations with n unknowns, we call it a
square n-by-n system.
We also can have an overdetermined system having more equations then unknowns (
as a skinny system, or un underdetermined system with less equations then unkowns (
which is known as fat system.

) known
),

In the following section we will look at some solution methods for square systems only.

6.2 Solving Square n-by-n Systems

Let

represent the n variables of the following square system of n linear equations

It is assumed that all constants

are known and it is desired to determine the unknown
variables
. Using the matrix-vector representation , the set of equations may be expressed as

where A is the coefficient matrix, x is the vector of unknowns and b is the right-hand-side (rhs) vector
of known constants .
In case there is some confusion about this operation, note that the left-hand-side (lhs) involves an n
matrix multiplied by an n
n
(also a column vector).

Thus, the entire set of equations can now be written in matrix-vector form as

which is called a linear non-homogeneous square system of ( n

homogeneous system.

n) equations. When

we have a

The result is surprisingly simple looking, but it obscures the fact that each of the quantities involved is

multidimensional in nature. Nevertheless this type of representation lends itself to the handling of large
quantities of data. We will now show how a square nonhomogeneous system of ( 3 3) equations can
be solved using the following methods:
1. Matrix inversion.
2. Direct Elimination.
3. Gaussian elimination.
4. Gauss-Jordan reduction.
5. Cramer's rule.
We assume that the solution indeed does exist, thus the coefficient matrix A is invertible, so its
determinant
In section 6.4, we will consider the solution of homogeneous systems to be used
for computing eigenvalues and eigenvectors.

1. Matrix Inversion
Now, how do we solve the equation
? If this was an ordinary one-dimensional algebraic
equation like
we would simply write
However with matrix equations such as this, we
premultiply both sides of the equation by the inverse matrix
Performing this operation we have

Note that if we premultiply on the left we must also premultiply on the righ. Now, the product of a
square matrix and its inverse is the identity matrix, that is

Finally, it can readily be shown that the product of the identity matrix times a vector is the vector itself,
that is

Putting together the preceding several steps, we obtain

Thus the solution x to the matrix-vector equation

is determined by taking the inverse of the
square matrix A and multiply it by the vector b on the right.

Example 6.1 Solving a linear

) system using matrix inversion

Solve the following square 2-by-2 system using the inverse matrix and matrix algebra:

Check the solution.

Solution
Matrix notation:

Solution:

Matrix:

Determinant:

Inverse:

Solution:

Check:

Software help

Example 6.2
Given a physical system with three unknown varaibles
three linear equations

that can be described by the following

Solve for the three unknowns using matrix inversion and check the result.

Solution
The set of equations can be written in matrix-vector form as

or, more compactly as

where matrix A is the coefficient matrix, which is the same as already
introduced in Example 4.3. The vector
is the vector of unknowns and
is the known rhs vector of constants.
The solution to this matrix equation is

where the inverse matrix was already determined in Example 5.2. Therefore we can immediately write
down the solution as

thus,

and

Substituting the solution into the matrix vector equation should result in the rhs vector of constants,
thus

Software help

2. Direct Elimination
Small systems of equations can always be solved by direct elimination. A systematic way of direct
elimination is the method of Gauss. Using this method we transform the square system of equations to
an equivalent triangular system that has the same solution but is easier to solve. During this
transformation we may only use the following basic operations:
(1). Interchange two equations in place.
(2). Multiply an equation by a nonzero number.

(3). Add a multiple of an equation to another equation.

Example 6.3
Given The following square system of equations

Find
(a). The solution using the three basic operations mentioned above. Show the used operations, both on
the bracketed equations and on the vector-matrix form.
(b). Check the solution by substituting it into the square system.

Solution
(a).
Step 1 Add -2 times row 1 to row 2 and -3 times row 1 to row 3 to make the elements in the first
column below the main diagonal zero.

Step 2 Add -4/6 times the second row to the third in order to make the elements in the second row
below the main diagonal zero.

The system of equations is now written in so-called upper triangular form. The corresponding upper
triangular A matrix has only zero entries below the main diagonal. The solution to such a triangular
system can easily be found by solving the three equations from bottom to top as follows:
Third equation :

substitute into the second.

Second equation:

substitute into the first equation.

Fist equation:

Done.

The solution of the triangular system is called back substitution.

(b).
Solution
Substitution of the solutionn into the equations gives

Software help

3. Gaussian Elimination and Back Substitution

We still consider the same the square linear system consisting of n equations in n unknowns:

The Gaussian elimination method uses the so-called augmented matrix, which is the coefficient
matrix A augmented (extended) with rhs vector b, thus

Now, we try to change the A matrix augmented matrix into an upper triangular matrix by performing
the following elementary row operations on the complete augmented matrix:
1. Replace a row of the matrix by a nonzero multiple of the row
2. Replace a row by the sum of that row and multiple of another row
3. Interchange two rows
These elementary row operations are in essence the same as the three basic operations used for direct
eliminationen. After applying these elementary row operations, the corresponding augmented matrix is
then converted into a so-called row echelon or staircase form.

After this row echelon matrix is derived a so-called back substitution is performed to find the solution.
We give an example to show the mechanics.

Example 6.4
substitution

3) system using Gaussian elimination and back

Example 6.2)

Find the solution by Gaussian elimination and back substitution. Check the result.

Solution
Form the augmented matrix and write next to it the elementary row operation to perform.

[A|b] =

Add row 1 to row 2

Add (-3) times row 1 to row 3

Add (4/3) times row 2 to row 3

Stop: Upper triangular matrix

Equations

Solve system by performing a back substitution

Solve
Solve
Solve

Solution :

It took 3 elimination steps and a backsubstitution step to arrive here.

Software help

4. Gauss-Jordan Reduction
In this case, we try to reduce the matrix A part of the augmented matrix into an identity matrix using
elementary row operations only. Doing the elimination process, the vector b part automatically changes
to the solution. We use the same example to show the mechanics. Gauss-Jordan reduction is sometimes
shortly called row reduction.

Example 6.5
Given

Find the solution by Gauss-Jordan reduction.

Solution
To solve the given linear system by Gauss-Jordan reduction, we transform the A part of the last matrix
of the previous example, which is in row echelon form, into the identity matrix by the following steps
using elementary row operations only. (next to the matrix we again denote the applied operation):

Step 1

Multiply R3 by

Step 2

Add R3 to R1

Step 3

Add R3 -2 times to R2

Step 4

Divide R3 by 3

Step 5

Add R2 -2 times to R1

Step 6

So, we add 6 Gauss-Jordan reduction steps to the 3 Gaussian elimination steps to arrive at the result.
Notice that we now do not need the backsubstitution process.
Remarks
As we perform elementary row operations, we may encounter a row of the augmented matrix being
transformed to upper triangular form whose first n entries are zero and who's
entry is not zero.
In this case, we can stop our computation and conclude that the given linear system is inconsistent,
and therefore has no solution. We come back to this subject in section 6.4.
In both Gaussian elimination and Gauss-Jordan reduction, we can only use row operations. Do not
try to use any column operations.
Software help

5. Cramer's Rule

Cramer's rule is already introduced in section 3.2

Example 6.6
(a). Use Cramer's rule to solve

and check the result.

(b). How many determinants do you have to evaluate to find the solution.
(c). Comment on the amount of computation to perform compared to Gaussian elimination and back
substitution and ordinary elimination.

Solution
(a). Crame

hecking the result

In the same manner we find

Therefore
Substitution of the solution into the three equations shows that the solution is correct:

(b). Number of determinants to evaluate to find the solution

Cramer's rule gives us a convenient method for writing the solution to an n n linear system in terms of
determinants. To compute the solution, we must evaluate
determinants of order n. In this
example
so we must compute 4 determinants.
(c). Comment on the amount of computation to perform compared to Gaussian elimination or
ordinary elimination
Evaluating even two of these determinants generally involves more computation than solving the
system using straight forward Gaussian elimination.
Using ordinary elimination (which always works for a 3-by-3 system) we find:

Substitute
Substitute

:
and

Cramer's
rule
One of the advantages of Cramer's rule is that we immedaitely can see that the system has no
solution.
Software help

6.3 Grafical Solution of Square (

) Systems

If we have equations in
we can try a geometrical solution, using the row format or column format
description of the system, see also section 3.4. We give an example with three sets of equations in

Example 6.7

1.

Find the solution by using matrix inversion and explain the result graphically.

Solution
Use:

With:

|A|=
Ad 1. Unique solution case

Equations

Matrix Equation

Matrix

Determinant

Inverse matrix

Solution:

Grafical interpretation
Row format
One intersection = solution
Column format
Linear combination:

Column format

Row format
x

b
2

2
x = x K2
2

2
a

1.5$a

solution

K1

(1.5, -0,5)
K2

K0.5$a

x =Kx C 1
2

Fig. 15 Row format and Column format Equation Set 1

Ad2. No solution case

Equations:

Matrix Equation

Matrix:

Determinant:

singular matrix!

Grafical interpretation
Row format
Column format
Linear combination: Impossible since and
(because lying along the same carrier).

are not linearly independent

Column format

Row format

carrier
x

x
K1

1
b

K1

x =K2$x C1
2

x =K2$x K 0.5
2

Fig. 16 Row format and Column format Equation Set 2

Ad 3.Infinitly many solutions case

Equations:

Matrix Equation

Matrix:

Determinant:
Remark:

singular matrix!
Actually, we only have one equation :
Choosing
Hence x =

so infinitly many solutions

Grafical interpretation
Row format
Column format
Infinitly many linear combination possible since

and b ly along the same carrier.

0 many combinations possible!

For example

Row format

Column format
carrier
a1

a2
1
K1

b
0.5 1

0
K1

4$x C2$ x = 2
1

2$x Cx = 1
1

Fig. 17 Row format and Column format Equation Set 3

Software help

6.4 Solvability of Square Non-Homogeneous (n n) Systems

We have seen in the previous sections that a non-homogeneous square (n n) system
has a
unique solution if the determinant
so only for non-singular coefficient matrix A. The
system is then called consistent with one unique solution:

Keeping in mind the definition of rank, we also can say that the system has a unique solution if the
coefficient matrix A and the augmented matrix
both are full rank, that is if
.
However, if the coefficient matrix is singular, so if
then the solution may have infinitly
many solutions. or no solution at all. The system has infinitly many solutions if the coefficient matrix
A and the augmented matrix
are equal but not full rank, that is iff

The system then is still called consistent, however with infinitly many solutions. We can choose
so-called free variables to find an expression for the infinitly many solutions.
On the other hand, the system is inconsistent, hence has no soution, if
and the coefficient
matrix A and the augmented matrix
have different ranks, that is if

The solvability of a square non-homogeneou (n n) system

can be depicted as shown in the
figure below. The rank can, as expected, conveniently be computed in Matlab with rank(A) and in
Maple with

Non-homogeneous
n n System

Ax =b

A is singular
det A = 0

rank A = rank A b = r ! n
N solutions
n Kr free variables

consistent system

6.5 Square (

rank A s rank A b
0 solutions

inconsistent system

A is non-singular
det A s 0

1 solution

consistent system

) Systems with no Solutions or Infinitly Many Solutions

We know from the preceeding section that, a square system has only a unique solution if the
determinant of the coefficient matrix is nonsingular, thus if
Such a system is called
consistent. If however the determinant is singular, thus if
the square system may have no
solution or infinitly many solutions. A system with no solutions is called inconsistent. A system with
infinitly many solutions is also called consistent. We give an example to show how we can find out
what kind of solution we have using the flowchart above and just using Gaussian elimination.

Example 6.8
zero determiant

Given

Find
(a). Check that the determinant of the coefficient matrix A is zero.
(b). Show, that the system is inconsistent with no solutions using the flowchart.
(c). Show, that the system is inconsistent with no solutions using Gaussian elimination.
(d). The rhs vector of known constants is changed to
Repeat part (b) and (c), but now showing that the system has infinitly many solutions.
Show that the solution vector can be written as

Solution
(a). Check that the determinant of the coefficient matrix A is zero.
Expanding the determinant along the first row gives us:

(b). Show, that the system is inconsistent with no solutions using the flowchart.
leads to: 0 solutions.

Maple/Matlab:

Nonhomogeneous n n System

Ax =b

A is singular
det A = 0

rank A = rank A b = r ! n
N solutions
n Kr free variables

consistent system

rank A s rank A b
0 solutions

inconsistent system

A is non-singular
det A s 0

1 solution

consistent system

(c). Show, that the system is inconsistent with no solutions using Gaussian elimination.

Gaussian elimination on augmented matrix delivers

Add -3 times R1 to R2

Add -3 times R1 to R3

Add -2 times R2 to R3

Stop

As soon as we have a zero on the main diagonal of the A part of the augmented matrix and a non-zero rhs

value (-5) , the system is inconsistent .

To show the inconsistency, look at the resulting equations:

The last equation is impossible, hence we have an inconsistent square system with no solutions.
(d). The rhs vector of known constants is changed to
(d1) Show, that the system now has ininitly many solutions using the flowchart.
Maple/Matlab:

Nonhomogeneous n n System

Ax =b

A is singular
det A = 0

rank A = rank A b = r ! n
N solutions
n Kr free variables

consistent system

rank A s rank A b
0 solutions

inconsistent system

A is non-singular
det A s 0

1 solution

consistent system

(d2) Show, that the system has ininitly many solutions using Gaussian elimination.
Show that the solution vector can be written as

Gaussian elimiantion on the new augmented matrix:

Add -3 times R1 to R2

Add -3 times R1 to R3

Add -2 times R2 to R3

Stop

As soon as we have a all zero row, the system has infinitly many solutions.

To show the infinitly many solutions, look at the resulting equations:

We now have two equations in three unknows, so one variable can be freely choosen. This variable is
the so-called free variable. Any of the three variables can be picked as free variable. The other two then
can be expressed in the free one. In this case it is convenient to choose as free variable, see also the
Maple and Matlab solutions.
Picking

as a free variable, we have

and

We thus indeed have

infinitly many solutions because the free variable can have any real value. Look at the Maple and/or
Matlab solution to see why is choosen as free variable in this case.

The solution can thus be written as

and is called the general solution. A particular solution is found by selecting a particular value for

Software help

6.6 Linear Homogeneous Square System of (n n) Equations

A homogeneous n
and has the form

n square linear system is a special case of the non-homogeneous square system

that is the rhs vector of know constants is all zero, so

The resulting set of equations can now

briefly be written as

The homogeneous n n linear system has exactly one solution, namely the trivial solution
when A is nonsingular, that is when
Nontrivial solutions, including the zero vector, only exist when A is singular, that is when
In that case there are infinitly many solutions, depending on (
Notice is the number of variables and is the rank of A .

free to choose parameters.

Recall that the rank of a matrix A equals the number of independent column vectors of A. If all
column vectors of matrix
are independent, we say that the matrix has full rank and
More simply put, the rank of a matrix A is the number of non-zero rows of the with
rref(A)reduced matrix in Matlab or with
in Maple.
The rank can, as expected, conveniently be computed in Matlab with rank(A) and in Maple with
The criteria for the solvability of a homogeneous n
below.

n linear system are depicted in the block diagram

Homogeneous
n n System

Ax =0

A is singular
det A = 0

N solutions
including the trivial solution
n Kr free parameters

A is non-singular
det A s 0

1 solution x=0
(trivial solution)

Example 6.9 Homogeneous Square Linear System 1

Investigate the solvability of the following homogeneous linear system of equations.

Solution

You can easily verify, that

free variable.

and

so there must be infinitly many solutions with

Row reduction delivers

R =

x1
1
0
0

x2
0
1
0

x3
-0.2
-0.5
0.0

b
0
0
0

non-zero row
non-zero row
zero row

non-pivot row

Comment
Free variable is the one above the non-pivot row , so
Two nonzero rows indicate that rank(A) = 2
If

we choose

Then
And

A general solution vector with integer elements then would be :

Software help

Example 6.10 Homogeneous Square Linear System 2

Investigate the solvability of the following homogeneous linear system of equations:

Solution
As you can easily verify, the determinant
the trivial solution
Software help

7. Problems
1.

Given the matrices

Compute:
(a).
(b).
(c).
(d).

(if possible)

2.

Given the vectors x =

Determine x y and y x
3.
Given the matrices

and y =

so the only solution for this homogeneous system is

Compute if possible:
(a).
(b).
(c).
(d).
(e).

4. Determine the determinant and the inverse matrix of the following matrices using the "Adjoint
Formula":

5. The analysis of the forces in a basic static structural system involve some equilibrium equations:
(a). The algebraic sum of all the forces in any direction is zero.
(b). The algebraic sum of all moments about any pivot point is zero
The equations obtained for a certain two-dimensional system with the forces in Newtons [N] are:

Arrange the equations in matrix-vector form and solve for the three variables using matrix inversion
with software. Compute with minimal 4 decimals accuracy, but give the answer in two decimals.

6. The analysis of the mesh currents in a dc circuit utilizes Kirchhoff's voltage law (KVL), which
states that the algebraic sum of the voltages around each loop is zero. With voltages expressed interms
of three unknown mesh currents
and measured in ampres [A], the KVL equations for a certain
three-mesh electric circuit are:

Arrange the equations in matrix form and solve for the three mesh currents using matrix inversion with
software.

7. The analysis of the node voltages in a dc circuit utilizes Kirchhoff's current law (KCL), which
states that the algebraic sum of the currents leaving each node is zero. With currents expressed in terms
of three node voltages
and
measured in volts [V], the KCL equations for a certain electric
circuit are

Arrange the equations in matrix form and solve for the three node voltages using inversion with
software.

8. The equations involving a certain steel girder involve three forces

[N], and they are

and

measured in newtons

Arrange the equations in matrix form and solve for the three forces using matrix algebra. Check the
result by performing a simple elimination method. Compute with minimal 4 decimals. Write the results
using two decimals accuracy. You may use software for matrix inversion.

9. Solve, using Cramer's rule

Determine the solution also using Gaussian elimination or Gauss-Jordan reduction.

10.
Given matrix

(a). Compute
(b). Compute
(c). Explain why

. Is A singular? Why?
and the matrix product

11. Given A =

(a). Compute
. Is A singular? Why?
(b). Compute
and the matrix product
(c). Explain the result.

12. Given the linear system of equations

Find the solution using Gaussian elimination.

13. Given the same system as in Problem 12. Find the solution using Gauss-Jordan reduction.

14. Determine the inverse matrix of matrix

using elementary row operations only.

8. Solutions

1. Given the matrices

Compute:
(a).
(b).
(c).
(d).

(if possible)

Solution 1

2. Given the vectors x =

and y =

Determine x y and y x
Solution 2
3. Given the matrices
Compute if possible:
(a).
(b).
(c).
(d).
(e).
Solution 3
4. Determine the determinant and the inverse matrix of the following matrices using the "Adjoint
Formula":

Solution 4
5. The analysis of the forces in a basic static structural system involve these equilibrium equations:
(a). The algebraic sum of all the forces in any direction is zero.
(b). The algebraic sum of all moments about any pivot point is zero
The equatins obtained for a certain two-dimensional system with the forces in Newtons [N] are:

Arrange the equations in matrix-vector form and solve for the three variables using matrix inversion
with software. Compute with minimal 4 decimals accuracy, but give the answer in two decimals.
Solution 5
6. The analysis of the mesh currents in a dc circuit utilizes Kirchhoff's voltage law (KVL), which
states that the algebraic sum of the voltages around each loop is zero. With voltages expressed interms
of three unknown mesh currents
and measured in ampres [A], the KVL equations for a certain
three-mesh electric circuit are:

Arrange the equations in matrix form and solve for the three mesh currents using matrix inversion with
software.
Solution 6
7. The analysis of the node voltages in a dc circuit utilizes Kirchhoff's current law (KCL), which
states that the algebraic sum of the currents leaving each node is zero. With currents expressed in terms
of three node voltages
and
measured in volts [V], the KCL equations for a certain electric
circuit are

Arrange the equations in matrix form and solve for the three node voltages using matrix inversion with
software.
Solution 7
8. The equations involving a certain steel girder involve three forces
[N], and they are

and

measured in newtons

Arrange the equations in matrix form and solve for the three forces using matrix algebra. Check the
result by performing a simple elimination method. Compute with minimal 4 decimals. Write the results
using two decimals accuracy. You may use software for matrix inversion.
Solution 8
9. Solve, using Cramer's rule the following set of linear equations

Determine the solution also using Gaussian elimination or Gauss-Jordan reduction.

Solution 9
10.
Given matrix

(a). Compute
(b). Compute
(c). Explain why

. Is matrix A singular? Why?

and the matrix products

Solution 10

11. Given A =

(a). Compute
. Is A singular?
(b). Compute
and the matrix product
(c). Explain the result.

Solution 11
12. Given the linear system of equations

Find the solution using Gaussian elimination.

Solution 12
13. Given the same system as in Problem 12. Find the solution using Gauss-Jordan reduction.

Solution 13

14. Determine the inverse matrix of

Solution 14

End of Matrices and Vectors

using elementary row operations only.