Matrix (mathematics)

Specific entries of a matrix are often referenced by using pairs of subscripts.

In mathematics, a matrix (plural matrices, or less commonly matrixes) is a rectangular

array of numbers, such as

(OR)

A matrix is a rectangular array of elements. If it has m rows and n columns

then the matrix is called an m × n matrix. It is called the size of this matrix.
Let A be an m × n matrix, the element on the ith row and jth column is called
the (i, j)th entry of the matrix.
Notation: Let A be an m × n matrix. We will denote it by A= (ai j ) , where aij is
the (i, j)th entry of A.

Definition
A matrix is a rectangular arrangement of numbers.[1] For example,

An alternative notation uses large parentheses instead of box brackets:

The horizontal and vertical lines in a matrix are called rows and columns, respectively.
The numbers in the matrix are called its entries or its elements. To specify a matrix's size,
a matrix with m rows and n columns is called an m-by-n matrix or m × n matrix, while m
and n are called its dimensions. The above is a 4-by-3 matrix.

A matrix where one of the dimensions equals one is also called a vector, and may be
interpreted as an element of real coordinate space. An m × 1 matrix (one column and m
rows) is called a column vector and a 1 × n matrix (one row and n columns) is called a
row vector. For example, the second row vector of the above matrix A is

Notation

A common shorthand is A = [ai,j]i=1,...,m; j=1,...,n or more briefly A = [ai,j]m×n

to define an m × n matrix A. Usually the entries ai,j are defined separately for all integers
1 ≤ i ≤ m and 1 ≤ j ≤ n. They can however sometimes be given by one formula; for
example the 3-by-4 matrix

can alternatively be specified by A = [i − j]i=1,2,3; j=1,...,4.

Some programming languages start the numbering of rows and columns at zero, in which
case the entries of an m-by-n matrix are indexed by 0 ≤ i ≤ m − 1 and 0 ≤ j ≤ n − 1.[2] This
article follows the more common convention in mathematical writing where enumeration
starts from 1
EQUAL MATRICES:
Two matrices are said to be equal if they are of the same size and their (i, j)th
entries are all equal.

Basic operations(Algebraic operations on matrices)

Main articles: Matrix addition, Scalar multiplication, Transpose, and Row operations

There are a number of operations that can be applied to modify matrices called matrix
addition, scalar multiplication and transposition.[3] These form the basic techniques to
eal with matrices.

Definition: (Matrix addition) Let A = ( ai j ) , B = ( bi j ) be two m × n matrices of

numbers. Then, A + B is an m × n matrix such that its (i, j)th entry is ai j + bi j i.e.
( ai j ) + ( bi j ) = ( ai j + bi j ) .
 1 6 −1  2 3 3   3 9 2 
Example:  + = 
0 1 5   −1 0 1  −1 1 6 

Definition: (Scalar multiplication) Let A = ( ai j ) be an m × n matrix of numbers

and let a be another number. Then, kA , the scalar multiplication of the
number a to the matrix A , is an m × n matrix such that its (i, j)th entry is
k ⋅ ai j i.e. k ( ai j ) = ( k ⋅ ai j ) .

 1 −1 0   3 −3 0 
Example: 3  =  , 4 (1 3 5 7 ) = ( 4 12 20 28 ) .
2 5 7   6 15 21
2  4 
   
2 4 =  8 .
 6  12 
   

Remark: Let B = (bi j ) . Then, − B is defined to be (−bi j ) . Note that − B = (−1) B .

A − B is defined to be (ai j ) − (bi j ) = (ai j − bi j ) .
Addition and scalar multiplication satisfy the following laws.

Theorem: Let A, B,C be matrices of the same size and let k ,k ′ be numbers. Let
0 denote the zero matrix of the same size. Then,

(i) Associative Law for Addition: ( A + B ) + C = A + ( B + C )

(ii) Commutative Law for Addition: A + B = B + A
(iii) Neutral Element for Addition: A + 0 = 0 + A
(iv) Existence of Inverse: Let − A = (−1) A . Then, A + (− A) = (− A) + A = 0
(v) Distributive Law: k ( A + B) = kA + kB , (k + k ′) A = kA + k ′A
(vi) (kk ′) A = k (k ′A)
(vii) 1A = A .

Definition: (Matrix multiplication) Let A = ( aik ) be an m × r matrix and let

B = ( bpq ) be an r × n matrix. Then, we can define AB as an m × n matrix such
l =r
that its (i, j)th entry is equal to ai1b1 j + ai 2b2 j + L + air brj = l = ∑ ail blj .
l =1

Remark: We can form the product AB whenever the number of columns of

th
A equal to the number of rows of B . To compute the (i, j) entry of AB , we
use the ith row of A and the jth column of B .

 1 0 30 1 0 0  0 1 0 0
Example:  −1 2 2  1 −1 0 1 =  2 −3 0 4 
   
 0 5 1  0 0 0 1  5 −5 0 6 
    

Theorem: Let A, B,C be matrices. Then, whenever the products and sums are
defined,

(i) Associative Law: ( AB)C = A( BC )

(ii) Distributive Law: A( B + C ) = AB + AC , ( B + C ) A = BA + CA .
(iii) k ( AB ) = (kA) B = A(kB) , where k is a number.
Familiar properties of numbers extend to these operations of matrices: for example,
addition is commutative, i.e. the matrix sum does not depend on the order of the
summands: A + B = B + A.[4] The transpose is compatible with addition and scalar
multiplication, as expressed by (cA)T = c(AT) and (A + B)T = AT + BT. Finally, (AT)T = A.

Row operations are ways to change matrices. There are three types of row operations:
row switching, that is interchanging two rows of a matrix, row multiplication,
multiplying all entries of a row by a non-zero constant and finally row addition which
means adding a multiple of a row to another row. These row operations are used in a
number of ways including solving linear equations and finding inverses.

Row vector
In linear algebra, a row vector or row matrix is a 1 × n matrix, that is, a matrix
consisting of a single row:[1]

The transpose of a row vector is a column vector:

Square matrices
A square matrix is a matrix which has the same number of rows and columns. An n-by-n
matrix is known as a square matrix of order n.

1 5
  is a 2 × 2 square matrix.
7 2

Any two square matrices of the same order can be added and multiplied. A square matrix
A is called invertible or non-singular if there exists a matrix B such that

AB = In.[13]

This is equivalent to BA = In.[14] Moreover, if B exists, it is unique and is called the

inverse matrix of A, denoted A−1.
Diagonal matrix
In linear algebra, a diagonal matrix is a square matrix in which the entries outside the main
diagonal (↘) are all zero. The diagonal entries themselves may or may not be zero. Thus, the
matrix D = (di,j) with n columns and n rows is diagonal if:

For example, the following matrix is diagonal:

The term diagonal matrix may sometimes refer to a rectangular diagonal matrix, which is an m-
by-n matrix with only the entries of the form di,i possibly non-zero; for example,

, or

The entries Ai,i form the main diagonal of a matrix. The trace, tr(A) of a square matrix A
is the sum of its diagonal entries. While, as mentioned above, matrix multiplication is not
commutative, the trace of the product of two matrices is independent of the order of the
factors: tr(AB) = tr(BA).[15]

ZERO MATRIX:
Definition: The matrix with all entries equal to zero is called a zero matrix.
An n × n matrix (ai j ) such that ai j = 1 if i = j and ai j = 0 otherwise is called the
identity matrix of order n. It is usually denoted by I n .

0 0 0 0 0 1 0 0
  1 0  
Examples:  0 0 0 0 0  is the 3 × 5 zero matrix. I 2 =   , I3 =  0 1 0  .
0 0 0 0 0 0 1 0 0 1
   
Remark: We will not have special notation for zero matrix. Usually, we just
denote it by 0 . Its size is determined by the context.
Identity matrix
In linear algebra, the identity matrix or unit matrix of size n is the n-by-n square matrix with
ones on the main diagonal and zeros elsewhere. It is denoted by In, or simply by I if the size is
immaterial or can be trivially determined by the context. (In some fields, such as quantum
mechanics, the identity matrix is denoted by a boldface one, 1; otherwise it is identical to I.)

Some mathematics books use U and E to represent the Identity Matrix (meaning "Unit Matrix"
[1]
and "Elementary Matrix", or from the German "Einheitsmatrix", respectively), although I is
considered more universal.

The important property of matrix multiplication of identity matrix is that for m-by-n A

Involutory matrix
In mathematics, an involutory matrix is a matrix that is its own inverse. That is, matrix A is an
involution iff A2 = I. One of the three classes of elementary matrix is involutory, namely therow-
interchange elementary matrix. A special case of another class of elementary matrix, that which
represents multiplication of a row or column by −1, is also involutory; it is in fact a trivial example
of a signature matrix, all of which are involutory.

Involutory matrices are all square roots of the identity matrix. This is simply a consequence of the
fact that any nonsingular matrix multiplied by its inverse is the identity. If A is an n × nmatrix,
then A is involutory if and only if ½(A + I) is idempotent.

An involutory matrix which is also symmetric is an orthogonal matrix, and thus represents
an isometry (a linear transformation which preserves Euclidean distance). A reflection matrix is an
example of an involutory matrix.
Nilpotent matrix
In linear algebra, a nilpotent matrix is a square matrix N such that

for some positive integer k. The smallest such k is sometimes called

the degree of N.

Examples: The matrix

is nilpotent, since M2 = 0. More generally, any triangular matrix with 0's along the main
diagonal is nilpotent.

For example, the matrix

is nilpotent, with

Though the examples above have a large number of zero entries, a typical nilpotent
matrix does not. For example, the matrices

both square to zero, though neither matrix has zero entries.

Triangular matrix
In the mathematical discipline of linear algebra, a triangular matrix is a special kind of
square matrix where the entries either below or above the main diagonal are zero.
Because matrix equations with triangular matrices are easier to solve they are very
important in numerical analysis. The LU decomposition gives an algorithm to decompose
any invertible matrix A into a normed lower triangle matrix L and an upper triangle
matrix U.

A matrix of the form

is called lower triangular matrix or left triangular matrix, and analogously a matrix of
the form

is called upper triangular matrix or right triangular matrix.

If all entries outside the main diagonal are zero, A is called a diagonal matrix. If only all
entries above (below) the main diagonal are zero, A is called a lower triangular matrix
(upper triangular matrix, respectively). For example, if n = 3, they look like

(diagonal), (lower) and

(upper triangular matrix).
The standard operations on triangular matrices conveniently preserve the triangular form:
the sum and product of two upper triangular matrices is again upper triangular. The
inverse of an upper triangular matrix is also upper triangular, and of course we can
multiply an upper triangular matrix by a constant and it will still be upper triangular. This
means that the upper triangular matrices form a subalgebra of the ring of square matrices
for any given size. The analogous result holds for lower triangular matrices. Note,
however, that the product of a lower triangular with an upper triangular matrix does not
preserve triangularity.

Tridiagonal matrix
In linear algebra, a tridiagonal matrix is a matrix that is "almost" a diagonal matrix. To
be exact: a tridiagonal matrix has nonzero elements only in the main diagonal, the first
diagonal below this, and the first diagonal above the main diagonal.

For example, the following matrix is tridiagonal:

A determinant formed from a tridiagonal matrix is known as a continuant.[1

Transpose
In linear algebra, the transpose of a matrix A is another matrix AT (also written A′, Atr or
t
A) created by any one of the following equivalent actions:

• write the rows of A as the columns of AT

• write the columns of A as the rows of AT
• reflect A by its main diagonal (which starts from the top left) to obtain AT

Formally, the (i,j) element of AT is the (j,i) element of A.

[AT]ij = [A]ji

If A is a m × n matrix then AT is a n × m matrix. The transpose of a scalar is the same

scalar.

 1 7
 1 −1 2 
Example: Let A =  −1 0  . Then, AT =  .
 2 3  7 0 3
 
NOTE: Let A, B be matrices and k be a number. Then, whenever the sum and
product are defined,

(i) ( A + B )T = AT + BT
(ii) ( AT )T = A
(iii) (kA)T = kAT

(iv) ( AB)T = BT AT .

Symmetric matrix
In linear algebra, a symmetric matrix is a square matrix, A, that is equal to its transpose

The entries of a symmetric matrix are symmetric with respect to the main diagonal (top
left to bottom right). So if the entries are written as A = (aij), then

for all indices i and j. The following 3×3 matrix is symmetric:

A matrix is called skew-symmetric or antisymmetric if its transpose is the same as its

negative. The following 3×3 matrix is skew-symmetric:

Skew-symmetric matrix
In linear algebra, a skew-symmetric (or antisymmetric or antimetric[1]) matrix is a
square matrix A whose transpose is also its negative; that is, it satisfies the equation:

or in component form, if : for all and

For example, the following matrix is skew-symmetric:

Compare this with a symmetric matrix whose transpose is the same as the matrix

or an orthogonal matrix, the transpose of which is equal to its inverse:

The following matrix is neither symmetric nor skew-symmetric:

Every diagonal matrix is symmetric, since all off-diagonal entries are zero. Similarly,
each diagonal element of a skew-symmetric matrix must be zero, since each is its own
negative.

Orthogonal matrix
In linear algebra, an orthogonal matrix is a square matrix with real entries whose
columns (or rows) are orthogonal unit vectors (i.e., orthonormal). Because the columns
are unit vectors in addition to being orthogonal, some people use the term orthonormal
to describe such matrices.

Equivalently, a matrix Q is orthogonal if its transpose is equal to its inverse:

alternatively,
 (OR)

Definition: An n × n matrix A is called an orthogonal matrix whenever

AT A = I .
EXAMPLE:
 −1 0   1 0   −1 0   cos θ − sin θ 
 ,  ,  ,  
 0 −1  0 −1  0 1   sin θ cos θ 

Conjugate transpose

"Adjoint matrix" redirects here. An adjugate matrix is sometimes called a "classical adjoint matrix".

In mathematics, the conjugate transpose, Hermitian transpose, or adjoint matrix of an m-by-

*
n matrix A with complex entries is the n-by-m matrix A obtained from A by taking
thetranspose and then taking the complex conjugate of each entry (i.e. negating their imaginary
parts but not their real parts). The conjugate transpose is formally defined by

where the subscripts denote the i,j-th entry, for 1 ≤ i ≤ n and 1 ≤ j ≤ m, and the overbar denotes a
scalar complex conjugate. (The complex conjugate of a + bi, where a and b are reals, isa − bi.)

This definition can also be written as

where denotes the transpose and denotes the matrix with complex conjugated entries.

Other names for the conjugate transpose of a matrix are Hermitian conjugate, or transjugate.
The conjugate transpose of a matrix A can be denoted by any of these symbols:

or , commonly used in linear algebra

(sometimes pronounced "A dagger"), universally used in quantum mechanics

, although this symbol is more commonly used for the Moore-Penrose
pseudoinverse

In some contexts, denotes the matrix with complex conjugated entries, and thus the
conjugate transpose is denoted by or .
EXAMPLE:

then

Hermitian matrix

A Hermitian matrix (or self-adjoint matrix) is a square matrix with complex entries which is
equal to its own conjugate transpose – that is, the element in the ith row and jth column is equal
to the complex conjugate of the element in the jth row and ith column, for all indices i and j:

If the conjugate transpose of a matrix is denoted by , then the Hermitian property can be
written concisely as

Hermitian matrices can be understood as the complex extension of a real symmetric matrix.

For example,

is a Hermitian matrix

Skew-Hermitian matrix
In linear algebra, a square matrix with complex entries is said to be skew-Hermitian or
antihermitian if its conjugate transpose is equal to its negative.[1] That is, the matrix A is
skew-Hermitian if it satisfies the relation

where denotes the conjugate transpose of a matrix. In component form, this means that

for all i and j, where ai,j is the i,j-th entry of A, and the overline denotes complex
conjugation.
Skew-Hermitian matrices can be understood as the complex versions of real skew-
symmetric matrices, or as the matrix analogue of the purely imaginary numbers.[2]

Unitary matrix
In mathematics, a unitary matrix is an n by n complex matrix U satisfying the condition

where is the identity matrix in n dimensions and is the conjugate transpose (also
called the Hermitian adjoint) of U. Note this condition says that a matrix U is unitary if
and only if it has an inverse which is equal to its conjugate transpose

A unitary matrix in which all entries are real is an orthogonal matrix. Just as an
orthogonal matrix G preserves the (real) inner product of two real vectors,

so also a unitary matrix U satisfies

for all complex vectors x and y, where stands now for the standard inner product on
.

If is an n by n matrix then the following are all equivalent conditions:

1. is unitary
2. is unitary
3. the columns of form an orthonormal basis of with respect to this inner
product
4. the rows of form an orthonormal basis of with respect to this inner product
5. is an isometry with respect to the norm from this inner product
6. U is a normal matrix with eigenvalues lying on the unit circle.
Row operations :
Let AX = b be a system of m equations in n unknowns. To solve this system,
we apply three basic operations on the system of equations to get an
equivalent but simpler system. The three operations are

(i) Multiply an equation by a nonzero number

(ii) Interchange two equations
(iii) Add to a certain equation a multiple of another equation.

(OR)

Three Elementary Row Operations

1. Switch any tow rows.

2. Multiply a row by a nonzero constant.

3. Replace a row by the sum of that row and a multiple of another row.

Two matrices are called row equivalent if one can be transformed into the other using a
sequence of row operations. Since row operations do not effect the solution space, any
two row equivalent matrices have the same solution space.

Elementary matrix
In mathematics, an elementary matrix is a simple matrix which differs from the identity
matrix in a minimal way. The elementary matrices generate the general linear
group of invertible matrices, and left (respectively, right) multiplication by an elementary
matrix representelementary row operations (respectively, elementary column operations).

In algebraic K-theory, "elementary matrices" refers only to the row-addition matrices.

Use in solving systems of equations
Elementary row operations do not change the solution set of the system of linear
equations represented by a matrix, and are used in Gaussian
elimination (respectively, Gauss-Jordan elimination) to reduce a matrix to row echelon
form (respectively, reduced row echelon form).

The acronym "ero" is commonly used for "elementary row operations".

Operations
There are three types of elementary matrices, which correspond to three types of row
operations (respectively, column operations):

Row switching
A row within the matrix can be switched with another row.

Row multiplication
Each element in a row can be multiplied by a non-zero constant.

Row addition
A row can be replaced by the sum of that row and a multiple of another row.

The elementary matrix for any row operation is obtained by executing the operation on an identity
matrix.

(OR)

Definition: On any given matrix, the elementary row operations are listed in
the following.

(i) Multiply a certain row by a nonzero number.

(ii) Interchange two rows.
(iii) Adding to a certain row a multiple of another row.

Remark: Elementary row operations are reversible.

Example

Use the elementary row operations to put the following in rref.

Solution

We follow the procedures:

Rank (linear algebra)

The column rank of a matrix A is the maximal number of linearly independent columns of A.
Likewise, the row rank is the maximal number of linearly independent rows of A.

Since the column rank and the row rank are always equal, they are simply called the rank of A.

Alternative definitions
column rank – dimension of column space

The maximal number of linearly independent columns of the m×n matrix A with
entries in the field F is equal to the dimension of the column space of A (the column space being
m
the subspace of F generated by the columns of A, which is in fact just the image of A as a linear
map).
row rank – dimension of row space

Since the column rank and the row rank are the same, we can also define the rank of A as the
dimension of the row space of A, or the number of rows in a basis of the row space.
determinantal rank – size of largest non-vanishing minor

Another equivalent definition of the rank of a matrix is the greatest order of any non-zero minor in
the matrix (the order of a minor being the size of the square sub-matrix of which it is the
determinant). Like the decomposition rank characterization, this does not give an efficient way of
computing the rank, but it is useful theoretically: a single non-zero minor witnesses a lower bound
(namely its order) for the rank of the matrix, which can be useful to prove that certain operations
do not lower the rank of a matrix.

Properties
We assume that A is an m-by-n matrix over either the real numbers or the complex numbers, and
we define the linear map f by f(x) = Ax as above.

1. only a zero matrix has rank zero.

2. .

3. f is injective if and only if A has rank n (in this case, we say that A has full column rank).

4. f is surjective if and only if A has rank m (in this case, we say that A has full row rank).

5. In the case of a square matrix A (i.e., m = n), then A is invertible if and only if A has
rank n (that is, A has full rank).

6. If B is any n-by-k matrix, then

As an example of the "<" case, consider the product

Both factors have rank 1, but the product has rank 0.

 7. If B is an n-by-k matrix with rank n, then

8. If C is an l-by-m matrix with rank m, then

9. The rank of A is equal to r if and only if there exists an invertible m-by-m matrix X and an

invertible n-by-n matrix Y such that

where Ir denotes the r-by-r identity matrix.

10. Sylvester’s rank inequality: If A is a m-by-n matrix and B n-by-k, then

[5]
.
This is a special case of the next inequality.

11. The inequality due to Frobenius: if AB, ABC and BC are defined, then
[6]
.

12. Subadditivity: when A and B are of the

same dimension. As a consequence, a rank-k matrix can be written as the sum of k rank-1
matrices, but not fewer.

13. The rank of a matrix plus the nullity of the matrix equals the number of columns of the matrix
(this is the "rank theorem" or the "rank-nullity theorem").

14. The rank of a matrix and the rank of its corresponding Gram matrix are equal

This can be shown by proving equality of their null spaces. Null space of the Gram matrix is given
T
by vectors x for which A Ax = 0. If this condition is fulfilled, also holds 0 = xTATAx = | Ax | 2.
[7]
This proof was adapted from.

15. If A * denotes the conjugate transpose of A (i.e., the adjoint of A), then
.
Row echelon form
A matrix is in reduced echelon form, if

1) in each row, the first nonzero entry is 1 and all the other entries in that column are
zero.

2) a row in which all the entries are zero, is below all those rows which have nonzero
entries.

3) if the rows r1 and r2 , r1 < r2 , have their first nonzero elements in the columns c1 and c2 ,
respectively, then c1 < c2 .

(OR)

Definition: Let A = (ai j ) be a matrix. A is said to be in row echelon form or

called an row echelon matrix whenever
(i) All zero rows are on the bottom.
(ii) For the ith row let ji be the first column with nonzero entry, then
j1<j2<j3<…
It is said to be in row reduced echelon form or called a row reduced echelon
matrix if in addition to (i), (ii) the following condition is satisfied.

(iii) For each nonzero row, the first nonzero entry from the left is 1 and
it is the only nonzero entry in its column.

(OR)

Definition: Given a matrix A, its rank is defined to be the number of nonzero

rows in its row echelon form.

(OR)

Definition

An m x n matrix is in reduced row echelon form if it satisfies the following

properties:
 1. All zero rows, if any, are at the bottom of the matrix

2. The first nonzero entry of each row is a one. This one is called the
leading one or the corner.

3. Each corner is to the right and below the preceding corner.

4. The columns containing a leading one have zeros in all other entries.

If only 1, 2, and 3 are satisfied, then the matrix is in row echelon form.

Examples of Echelon and Row reduced echelon matrices:

2 3 2 0 4 5 −6 
  1 2 3 0 1 3 0 0 4
0 0 1 1 −3 2 0     
0 0 0 0 0 6 2
, 0 0 1 ,  0 0 0 1 0 −3  . The first two are
0 0 0 0 0 0 0 1 2
     
0 0 0 0 0 0 0 
echelon matrices and the third one is a reduced echelon matrix.

Theorem: Given any matrix A , one can reduce this matrix into echelon form
or row reduced echelon form by applying elementary row operations.

Proposition 4. Every matrix A is equivalent to a unique matrix AR , which is in reduced

echelon form. The rank of A = the rank of AR = the number of nonzero rows in AR .

Proof. Omitted.

Proposition 5. If A is an n x n matrix, then

the rank of A = n <=> AR = I.

Proof.
1) AR = I => the rank of A = n (= the number of nonzero rows)
2) The rank of A = n => the rank of AR = n => AR = I.

Proposition 6. If A is an n x n matrix, then the following statements are equivalent

(i) A -1 exists

(ii) the rank of A is n

(iii) AR = I

(iv) the rows(columns) of A are linearly independent.

Proof. Follows from Proposition 5 and results of the section 1.4.. (computation of an
inverse: (A | I) ~ ··· ~ (AR | B), if AR = I, then B = A -1).

Proposition 1 => (ii) <=> (iv)

Example 1: Existence of an inverse

2) Solvability of linear systems of equations:

Proposition 7. A linear system of equations has solutions if and only the ranks of the
coefficient matrix and the augmented coefficient matrix coincide.

Example: Reduce the following matrix to row reduced echelon form.

 2 2 −1 6 4   2 2 −1 6 4   2 2 −1 6 4 
     
 4 4 1 10 13  →  0 0 3 −2 5  →  0 0 3 −2 5  →
 6 6 0 20 19   6 6 0 20 19   0 0 3 2 7 
     

 2 2 −1 6 4   2 2 −1 6 4   2 2 −1 0 1
     
→  0 0 3 −2 5  →  0 0 3 −2 5  →  0 0 3 −2 5  →
 0 0 0 4 2   0 0 0 2 1  0 0 0 2 1
     

 2 2 −1 0 1  2 2 0 0 3   1 1 0 0 3 2 
     
→  0 0 3 0 6 →  0 0 3 0 6 →  0 0 3 0 6 →
 0 0 0 2 1  0 0 0 2 1  0 0 0 2 1
    

 1 1 0 0 3 2  1 1 0 0 3 2
   
→ 0 0 1 0 2 → 0 0 1 0 2 .
0 0 0 2 1  0 0 0 1 1 2 


Examples
This matrix is in reduced row echelon form:
The following matrix is also in row echelon form, but not in reduced row form:

However, the matrix below is not in row echelon form, as the leading coefficient of row 3 is not
strictly to the right of the leading coefficient of row 2, and the main diagonal is not made up of only
ones:

 1 2 −2 3 2 
Example: Find the rank of  2 4 −3 4 5  .
 5 10 −8 11 12 
 

 1 2 0 −1 4 
Solution: Since the reduced echelon form is  0 0 1 −2 1 . The rank is
0 0 0 0 0
 
then equal to 2.

Non-uniqueness
Every non-zero matrix can be reduced to an infinite number of echelon forms (they can all be
multiples of each other, for example) via elementary matrix transformations. However, all
matrices and their row echelon forms correspond to exactly one matrix in reduced row echelon
form.
Systems of linear equations
A system of linear equations is said to be in row echelon form if its augmented matrix is
in row echelon form. Similarly, a system of equations is said to be in reduced row echelon
form or canonical form if its augmented matrix is in reduced row echelon form.

Augmented matrix
Definition: A is called the coefficient matrix and the m × (n + 1) matrix ( A b )
is called the augmented matrix of the system of linear equations.
In linear algebra, the augmented matrix of a matrix is obtained by changing a matrix in some
way.

Given the matrices A and B, where:

Then, the augmented matrix (A|B) is written as:

This is useful when solving systems of linear equations or the augmented matrix may also be
used to find the inverse of a matrix by combining it with the identity matrix.

Gaussian elimination
In linear algebra, Gaussian elimination is an algorithm for solving systems of linear
equations, finding the rank of a matrix, and calculating the inverse of an invertible square
matrix. Gaussian elimination is named after German mathematician and scientist Carl
Friedrich Gauss.

Elementary row operations are used to reduce a matrix to row echelon form. Gauss–
Jordan elimination, an extension of this algorithm, reduces the matrix further to reduced
row echelon form. Gaussian elimination alone is sufficient for many applications.
Gaussian elimination: Let AX = b be the matrix representation of a system
of m equations in n unknowns. Apply elementary row operations to the
augmented matrix ( A b ) to reduce it to row reduced echelon form. If the
first nonzero entry of a nonzero row is on the last column then the system
has no solution. Otherwise, we can apply backward substitution to get
solutions of the system.

Example:1. Solve the following system by applying elementary row

operations to the augmented matrix.
x + 2 y − 2 z + 3w = 2
2 x + 4 y − 3 z + 4w = 5
5 x + 10 y − 8 z + 11w = 12

Solution:
 1 2 −2 3 2 
The augmented matrix is  2 4 −3 4 5  . In the following, we apply
 5 10 −8 11 12 
 
elementary row operations to this matrix reducing it to reduced echelon
form.
 1 2 −2 3 2   1 2 −2 3 2   1 2 −2 3 2
     
 2 4 −3 4 5  →  0 0 1 −2 1 →  0 0 1 −2 1 →
 5 10 −8 11 12   0 0 2 −4 2   0 0 0 0 0 
     

1 2 0 −1 4 
 
0 0 1 −2 1 . This is the augmented matrix of the following system
0 0 0 0 0 

x + 2y − w = 4 x = 4 − 2y + w
. Hence,
z − 2w = 1 z = 1 + 2w

Answer: x = 4 − 2 s + t , y = s, z = 1 + 2t , w = t , where s, t are arbitrary.

Example:2. In practice, one does not usually deal with the systems in terms of equations but
instead makes use of the augmented matrix (which is also suitable for computer manipulations).
This, then, is the Gaussian Elimination algorithm applied to the augmented matrix of the system
above, beginning with:

which, at the end of the first part of the algorithm, looks like this:

That is, it is in row echelon form.

At the end of the algorithm, if the Gauss–Jordan elimination is applied:

That is, it is in reduced row echelon form, or row canonical form.

Gauss–Jordan elimination
In linear algebra, Gauss–Jordan elimination is a version of Gaussian elimination that puts
zeros both above and below each pivot element as it goes from the top row of the given matrix
to the bottom. In other words, Gauss–Jordan elimination brings a matrix to reduced row
echelon form, whereas Gaussian elimination takes it only as far as row echelon form. Every
matrix has a reduced row echelon form, and this algorithm is guaranteed to produce it.

Gauss–Jordan elimination is considerably less efficient than Gaussian elimination

with backsubstitution when solving a system of linear equations. However, it is well suited for
calculating the matrix inverse.

It is named in after Carl Friedrich Gauss and Wilhelm Jordan, because it is a modification of
Gaussian elimination as described by Jordan in 1887. However, the method also appears in an
article by Clasen published in the same year. Jordan and Clasen probably discovered Gauss–
Jordan elimination independently (Althoen & McLaughlin 1987).
Finding the inverse of a matrix

Suppose A is a matrix and you need to calculate its inverse. The identity
matrix is augmented to the right of A, forming a matrix (the block matrix B =
[A,I]). Through application of elementary row operations and the Gaussian elimination
algorithm, the left block of B can be reduced to the identity matrix I, which leaves A − 1 in
the right block of B.

If the algorithm is unable to reduce A to triangular form, then A is not invertible.

In practice, inverting a matrix is rarely required. Most of the time, one is really after the
solution of a particular system of linear equations.[3]

Application to finding inverses

If Gauss–Jordan elimination is applied on a square matrix, it can be used to calculate the
matrix's inverse. This can be done by augmenting the square matrix with the identity matrix of the
same dimensions, and through the following matrix operations:

If the original square matrix, A, is given by the following expression:

Then, after augmenting by the identity, the following is obtained:

By performing elementary row operations on the [AI] matrix until it reaches reduced row echelon
form, the following is the final result:

The matrix augmentation can now be undone, which gives the following:
 or

A matrix is non-singular (meaning that it has an inverse matrix) if and only if the identity matrix
can be obtained using only elementary row operations.

System of linear equations

General form
A general system of m linear equations with n unknowns can be written as

Here are the unknowns, are the coefficients of the

system, and are the constant terms.

Often the coefficients and unknowns are real or complex numbers, but integers and rational
numbers are also seen, as are polynomials and elements of an abstract algebraic structure.

Matrix equation
The vector equation is equivalent to a matrix equation of the form

where A is an m×n matrix, x is a column vector with n entries, and b is a column vector
with m entries.

The number of vectors in a basis for the span is now expressed as the rank of the matrix.
Solution set

The solution set for the equationsx − y = −1 and 3x + y = 9 is the single point (2, 3).

A solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that
each of the equations is satisfied. The set of all possible solutions is called the solution set.

A linear system may behave in any one of three possible ways:

1. The system has infinitely many solutions.

2. The system has a single unique solution.
3. The system has no solution.

Consistency

The equations 3x + 2y = 6 and3x + 2y = 12 are inconsistent.

The equations of a linear system are consistent if they possess a common solution,
and inconsistentotherwise. When the equations are inconsistent, it is possible to derive
a contradiction from the equations, such as the statement that 0 = 1.

For example, the equations

are inconsistent. In attempting to find a solution, we tacitly assume that there is a solution; that is,
we assume that the value of x in the first equation must be the same as the value of x in the
second equation (the same is assumed to simultaneously be true for the value of y in both
equations). Applying the substitution property (for 3x+2y) yields the equation 6 = 12, which is a
false statement. This therefore contradicts our assumption that the system had a solution and we
conclude that our assumption was false; that is, the system in fact has no solution. The graphs
of these equations on the xy-plane are a pair ofparallel lines.

It is possible for three linear equations to be inconsistent, even though any two of the equations
are consistent together. For example, the equations

are inconsistent. Adding the first two equations together gives 3x + 2y = 2, which can be
subtracted from the third equation to yield 0 = 1. Note that any two of these equations have a
common solution. The same phenomenon can occur for any number of equations.

In general, inconsistencies occur if the left-hand sides of the equations in a system are linearly
dependent, and the constant terms do not satisfy the dependence relation. A system of equations
whose left-hand sides are linearly independent is always consistent.

System of Equations
Matrix algebra is used for solving systems of equations.
Can you illustrate this concept?
Matrix algebra is used to solve a system of simultaneous linear equations. In fact, for
many mathematical procedures such as the solution to a set of nonlinear equations,
interpolation, integration, and differential equations, the solutions reduce to a set of
simultaneous linear equations. Let us illustrate with an example for interpolation.

A general set of m linear equations and n unknowns,

a11 x1 + a12 x 2 + LL + a1n x n = c1
a 21 x1 + a 22 x2 + LL + a 2 n xn = c2
……………………………………
…………………………………….
a m1 x1 + a m 2 x2 + ........ + a mn x n = c m
can be rewritten in the matrix form as
 a11 a12 . . a1n   x1   c1 
a a 22 . . a 2 n   x 2   c 2 
 21
 M M  ⋅ = ⋅ 
    
 M M ⋅  ⋅ 
a m1 a m 2 . . a mn   x n  c m 
Denoting the matrices by [A] , [X ] , and [C ] , the system of equation is
[A][X ] = [C ] , where [A] is called the coefficient matrix, [C ] is called the right hand side
vector and [X ] is called the solution vector.
Sometimes [A][X ] = [C ] systems of equations are written in the augmented form. That is
a a12 ...... a1n M c1 
 11 
 a 21 a 22 ...... a 2 n M c 2 
[AMC ] =  M M 
 
 M M 
a m1 a m 2 ...... a mn M c n 

A system of equations can be consistent or inconsistent.

What does that mean?
A system of equations [A][X ] = [C ] is consistent if there is a solution, and it is
inconsistent if there is no solution. However, a consistent system of equations does not
mean a unique solution, that is, a consistent system of equations may have a unique
solution or infinite solutions (Figure 1).

[A][X]= [B]

Consistent System Inconsistent System

Unique Solution Infinite Solutions

Figure 5.1. Consistent and inconsistent system of equations flow chart.

Example 1
Give examples of consistent and inconsistent system of equations.

Solution
a) The system of equations
 2 4  x  6 
1 3   y  =  4 
    
is a consistent system of equations as it has a unique solution, that is,
 x  1
 y  = 1 .
  
b) The system of equations
 2 4   x  6 
 1 2   y  =  3
    
is also a consistent system of equations but it has infinite solutions as given as follows.
Expanding the above set of equations,
2x + 4 y = 6
x + 2y = 3
you can see that they are the same equation. Hence, any combination of ( x, y ) that
satisfies
2x + 4 y = 6
is a solution. For example ( x, y ) = (1,1) is a solution. Other solutions include
(x, y ) = (0.5,1.25) , (x, y ) = (0, 1.5) , and so on.
c) The system of equations
 2 4  x  6 
1 2   y  =  4 
    
is inconsistent as no solution exists.
How can one distinguish between a consistent and
inconsistent system of equations?
A system of equations [A][X ] = [C ] is consistent if the rank of A is equal to the rank of
the augmented matrix [AMC ]
A system of equations [A][X ] = [C ] is inconsistent if the rank of A is less than the rank
of the augmented matrix [AMC ] .
But, what do you mean by rank of a matrix?

Rank
The rank of a matrix is defined as the order of the largest square submatrix whose
determinant is not zero.
(OR)

The rank of any matrix A, written as rank(A) is the size of the largest nonvanishing
determinant contained in A.
2 3 2 3
Example : = − 12 ≠ 0 rank  =2
8 6 8 6 
3 6 3 6
=0 rank   =1
2 4 2 4 
Equivalently, then rank is the (maximum) number of linearly independent rows (or
columns) of matrix A, where a1 , a 2 ,..., a n is a set of linearly independent vectors iff
n
∑ k j a j = 0 implies k1 = k 2 = ⋅ ⋅ ⋅ = k n = 0 .
j =1

Example:
2 3 2  3 2 3  k1 
  k1   + k 2   = 0 ⇒    = 0 ⇒ k1 = 0, k 2 = 0
8 6  8   6 8 6  k 2 
3 6 3 6 3 6   k1 
  k1   + k 2   = 0 ⇒    = 0 ⇒ k1 = −2k 2
2 4  2 4 2 4  k 2 

Properties of the rank of a matrix:

If A~ m×n, 0 ≤ rank(A) = integer ≤ min(m, n).
If I~n×n, rank(I) = n, rank(0) = 0.
rank(A) = n, if A is an orthogonal matrix of dimension n.
rank(A) = rank( A′ ) = rank( A′A ) = rank( AA′ ).
If A and B are of the same order, rank(A+B )≤ rank(A)+rank(B).
If AB is defined, rank(AB)≤min[rank(A), rank(B)].
If A is diagonal, rank(A)= number of nonzero elements.
If A is idempotent, rank(A) = tr(A).
The rank of a matrix is not changed if one row (column) is multiplied by a nonzero
constant or if such a multiple of one row (column) is added to another row(column) .
A square matrix of order n is nonsingular iff it is of full rank
rank(A) = n or, equivalently, | A |≠0

The rank of a matrix is unchanged by premultiplying or postmultiplying by a

nonsingular matrix. Thus if there are nonsingular matrices E and F for which
 I 0
EAF =  , I ~ k × k ⇒ rank( A) = k
0 0

Example 2
What is the rank of
 3 1 2
[A] = 2 0 5 ?
1 2 3

Solution
The largest square submatrix possible is of order 3 and that is [ A] itself. Since
det( A) = −23 ≠ 0, the rank of [ A] = 3.

Example 3
What is the rank of
 3 1 2
[A] = 2 0 5 ?
5 1 7

Solution
The largest square submatrix of [ A] is of order 3 and that is [ A] itself. Since det( A) = 0 ,
the rank of [ A] is less than 3. The next largest square submatrix would be a 2 × 2 matrix.
One of the square submatrices of [ A] is

[B] =  
3 1
2 0
and det( B ) = −2 ≠ 0 . Hence the rank of [ A] is 2. There is no need to look at other 2 × 2
submatrices to establish that the rank of [A] is 2.
Example 4
How do I now use the concept of rank to find if
 25 5 1  x1  106.8 
 64 8 1  x  = 177.2 
  2   
144 12 1  x3  279.2
is a consistent or inconsistent system of equations?

Solution
The coefficient matrix is
 25 5 1
[A] =  64 8 1
144 12 1
and the right hand side vector is
106.8 
[C ] = 177.2 
279.2
The augmented matrix is
 25 5 1 M 106.8 
[B] =  64 8 1 M 177.2 
144 12 1 M 279.2
Since there are no square submatrices of order 4 as [B] is a 3 × 4 matrix, the rank of [B]
is at most 3. So let us look at the square submatrices of [B ] of order 3; if any of these
square submatrices have determinant not equal to zero, then the rank is 3. For example, a
submatrix of the augmented matrix [B ] is
 25 5 1
[D] =  64 8 1
144 12 1
has det( D ) = −84 ≠ 0 .
Hence the rank of the augmented matrix [B] is 3. Since [ A] = [ D] , the rank of [A] is 3.
Since the rank of the augmented matrix [B ] equals the rank of the coefficient matrix [ A] ,
the system of equations is consistent.
Example 5
Use the concept of rank of matrix to find if
25 5 1   x1  106.8 
64 8 1   x  = 177.2 
   2  
89 13 2  x3  284.0
is consistent or inconsistent?

Solution
The coefficient matrix is given by
25 5 1
[A] = 64 8 1
89 13 2
and the right hand side
106.8 
[C ] = 177.2 
284.0
The augmented matrix is
25 5 1 : 106.8 
[B] = 64 8 1 : 177.2 
89 13 2 : 284.0
Since there are no square submatrices of order 4 as [B] is a 4 × 3 matrix, the rank of the
augmented [B ] is at most 3. So let us look at square submatrices of the augmented
matrix [B] of order 3 and see if any of these have determinants not equal to zero. For
example, a square submatrix of the augmented matrix [B] is
25 5 1
[D] = 64 8 1
89 13 2
has det( D) = 0 . This means, we need to explore other square submatrices of order 3 of
the augmented matrix [B] and find their determinants.
That is,
 5 1 106.8 
[E ] =  8 1 177.2 
13 2 284.0
det( E ) = 0
25 5 106.8 
[F ] = 64 8 177.2 
89 13 284.0
det( F ) = 0

25 1 106.8 
[G ] = 64 1 177.2 
89 2 284.0
det(G ) = 0
All the square submatrices of order 3 × 3 of the augmented matrix [B ] have a zero
determinant. So the rank of the augmented matrix [B] is less than 3. Is the rank of
augmented matrix [B] equal to 2?. One of the 2 × 2 submatrices of the augmented
matrix [B] is

[H ] = 
25 5

64 8
and
det( H ) = −120 ≠ 0
So the rank of the augmented matrix [ B ] is 2.
Now we need to find the rank of the coefficient matrix [ B] .
25 5 1 
[A] = 64 8 1
89 13 2
and
det( A) = 0
So the rank of the coefficient matrix [ A] is less than 3. A square submatrix of the
coefficient matrix [ A] is

[J ] =  
5 1
8 1
det( J ) = −3 ≠ 0
So the rank of the coefficient matrix [ A] is 2.
Hence, rank of the coefficient matrix [ A] equals the rank of the augmented matrix [B].
So the system of equations [ A][ X ] = [C ] is consistent.
Example 6
Use the concept of rank to find if
 25 5 1  x1  106.8 
64 8 1  x  = 177.2 
   2  
89 13 2  x3  280.0
is consistent or inconsistent.

Solution
The augmented matrix is
25 5 1 : 106.8 
[B] = 64 8 1 : 177.2 
89 13 2 : 280.0
Since there are no square submatrices of order 4 × 4 as the augmented matrix [B] is a
4 × 3 matrix, the rank of the augmented matrix [B ] is at most 3. So let us look at square
submatrices of the augmented matrix (B) of order 3 and see if any of the 3 × 3 submatrices
have a determinant not equal to zero. For example, a square submatrix of order 3 × 3 of
[B]
25 5 1
[D] = 64 8 1
89 13 2
det(D) = 0
So it means, we need to explore other square submatrices of the augmented matrix [B ]
 5 1 106.8 
[E ] =  8 1 177.2 
13 2 280.0
det( E 0 ≠ 12.0 ≠ 0 .
So the rank of the augmented matrix [ B] is 3.
The rank of the coefficient matrix [ A] is 2 from the previous example.
Since the rank of the coefficient matrix [ A] is less than the rank of the augmented
matrix [B ] , the system of equations is inconsistent. Hence, no solution exists for
[ A][ X ] = [C ] .
If a solution exists, how do we know whether it is
unique?
In a system of equations [ A][ X ] = [C ] that is consistent, the rank of the coefficient
matrix [ A] is the same as the augmented matrix [ A C ] . If in addition, the rank of the
coefficient matrix [ A] is same as the number of unknowns, then the solution is unique; if
the rank of the coefficient matrix [ A] is less than the number of unknowns, then infinite
solutions exist.
[A] [X] = [B]

Consistent System if Inconsistent System if

rank (A) = rank (A.B) rank (A) < rank (A.B)

Unique solution if Infinite solutions if

rank (A) = number of unknowns rank (A) < number of unknowns

Figure 5.2. Flow chart of conditions for consistent and inconsistent system of
equations.

Example 7
We found that the following system of equations
 25 5 1  x1  106.8 
 64 8 1  x  = 177.2 
   2  
144 12 1  x3  279.2
is a consistent system of equations. Does the system of equations have a unique solution
or does it have infinite solutions?

Solution
The coefficient matrix is
 25 5 1 106.8 
[A] =  64 8 1 and the right hand side is [C ] = 177.2 
144 12 1 279.2
While finding out whether the above equations were consistent in an earlier example, we
found that the rank of the coefficient matrix (A) equals rank of augmented matrix [ AM C ]
equals 3.
The solution is unique as the number of unknowns = 3 = rank of (A).
Example 8
We found that the following system of equations
25 5 1   x1  106.8 
64 8 1   x  = 177.2 
   2  
89 13 2  x3  284.0
is a consistent system of equations. Is the solution unique or does it have infinite
solutions.

Solution
While finding out whether the above equations were consistent, we found that the rank of
the coefficient matrix [ A] equals the rank of augmented matrix ( AMC ) equals 2
Since the rank of [ A] = 2 < number of unknowns = 3, infinite solutions exist.

If we have more equations than unknowns in [A] [X] =

[C], does it mean the system is inconsistent?
No, it depends on the rank of the augmented matrix [AMC ] and the rank of [ A] .
a) For example
 25 5 1  106.8 
 64 8 1   x1  177.2 
   x2  =  
144 12 1    279.2
   x3   
 89 13 2 284.0
is consistent, since
rank of augmented matrix = 3
rank of coefficient matrix = 3
Now since the rank of (A) = 3 = number of unknowns, the solution is not only consistent
but also unique.
b) For example
 25 5 1  106.8 
 64 8 1   x1  177.2 
  x  =  
144 12 1   2  279.2
   x3   
 89 13 2 280.0
is inconsistent, since
rank of augmented matrix = 4
rank of coefficient matrix = 3
c) For example
25 5 1  106.8 
64 8 1   x1  177.2 
  x  =  
50 10 2  2  213.6
   x3   
89 13 2 280.0
is consistent, since
rank of augmented matrix = 2
rank of coefficient matrix = 2
But since the rank of [A] = 2 < the number of unknowns = 3, infinite solutions exist.

Example 9
Determine if
3 2  − 3 2 
[B ] =   is the inverse of [A] =  
5 3  5 − 3

Solution
3 2 − 3 2  1 0
[ B ][ A] =    =  = [I ]
5 3  5 − 3 0 1
Since
[ B ][ A] = [ I ] ,

[B] is the inverse of [A] and [ A] is the inverse of [B] .

But, we can also show that

− 3 2  3 2 1 0
[ A][ B] =   5 3 = 0 1 = [ I]
 5 − 3    

to show that [ A] is the inverse of [B] .

Can I use the concept of the inverse of a matrix to find
the solution of a set of equations [A] [X] = [C]?
Yes, if the number of equations is the same as the number of unknowns, the coefficient
matrix [ A] is a square matrix.
Given
[ A][ X ] = [C ]
Then, if [ A]−1 exists, multiplying both sides by [ A]−1 .
[ A] −1 [ A][ X ] = [ A]−1 [C ]
[ I ][ X ] = [ A] −1[C ]
[ X ] = [ A]−1 [C ]
This implies that if we are able to find [ A]−1 , the solution vector of [ A][ X ] = [C ] is
simply a multiplication of [ A]−1 and the right hand side vector, [C ] .

How do I find the inverse of a matrix?

If [ A] is a n × n matrix, then [ A]−1 is a n × n matrix and according to the definition of
inverse of a matrix
[ A][ A] −1 = [ I ]
Denoting
 a11 a12 ⋅ ⋅ a1n 
a 
 21 a 22 ⋅ ⋅ a 2 n 
[ A] =  ⋅ ⋅ ⋅ ⋅ ⋅ 
 
 ⋅ ⋅ ⋅ ⋅ ⋅ 
a n1 a n 2 ⋅ ⋅ a nn 
 a11' a12' ⋅ ⋅ a1' n 
 ' ' 
a 21 a 22 ⋅ ⋅ a 2 n 
'

[ A] −1 =  ⋅ ⋅ ⋅ ⋅ ⋅ 
 
 ⋅ ⋅ ⋅ ⋅ ⋅ 
a ' a ' ⋅ ⋅ a nn' 
 n1 n2 
 1 0 ⋅ ⋅ ⋅ 0 
0 1 0

0 ⋅ ⋅
[I ] =  
⋅ 1 ⋅
⋅ ⋅ ⋅
 
0 ⋅ ⋅ ⋅ ⋅ 1
Using the definition of matrix multiplication, the first column of the [ A] −1 matrix can
then be found by solving
 a11 a12 ⋅ ⋅ a1n   a11'  1
a  '   
a 22 ⋅ ⋅ a 2 n  a 21   0
 21 
 ⋅ ⋅ ⋅ ⋅ ⋅   ⋅  = ⋅
    
 ⋅ ⋅ ⋅ ⋅ ⋅   ⋅  ⋅
an1 a n 2 ⋅ ⋅ a nn  a n' 1  0
Similarly, one can find the other columns of the [ A]−1 matrix by changing the right hand
side accordingly.
Definition: Let A be an n × n matrix. The matrix B such that BA = AB = I n is
called the inverse of A and will be denoted by A−1 . A is then said to be
invertible.

Remark: Given AX = b , if A is invertible then X = A−1b .

another way to find the inverse of a matrix?

For finding the inverse of small matrices, the inverse of an invertible matrix can be found
by
[A]−1 = 1 adj( A)
det ( A)
where
T
C11 C12 L C1n 
C C 22 C 2 n 
adj ( A) =  21
 M 
 
C n1 C n 2 L C nn 
where C ij are the cofactors of aij . The matrix
 C11 C12 L C1n 
C 
 21 C 22 L C 2 n 
 M M 
 
C n1 L L C nn 
itself is called the matrix of cofactors from [A]. Cofactors are defined in Chapter 4.
( OR)

Inverse
If A is a square, nonsingular matrix of order n, a unique inverse matrix A−1 of order n
exists, where A A−1 = A−1 A = I .
The inverse matrix can be calculated as
(C ij ) (−1) M ji
' (i + j )

A = =
−1

A A
where (C ij ) is the matrix of cofactors, and its transpose (C ij ) is called the adjoint
'

matrix.

 1 −1 
(C ij )' = 1  3 −1  =  1 2 
'
2 3 1
A =
−1
A=  
1 3  A 2 3  −3 2   − 
 3 3
1 3

 1 −1 
 2 3   1 0
A A = 
−1
 1 2  =  
 1 3  −  0 1
 3 3

Properties of inverse matrix

−1
I =I
1
( A−1 ) −1 = A, ( A′)−1 = ( A−1 )′, A = A =
−1 −1

A
( AB )−1 = B −1 A−1
assuming that both A and B are nonsingular and of the same order
A = A′ iff A is orthogonal ( AA′ = A′A = I ), in which case A−1 and A′ are also
−1

orthogonal
If A is nonsingular and symmetric, so is A−1
NOTE:

Theorem: Let A, B be invertible n × n matrices. Then AB is invertible. In fact,

( AB) −1 = B −1 A−1 .

Theorem: Let A be an invertible n × n matrix. Then AT is invertible. In fact,

( AT ) −1 = ( A−1 )T .

Example 10
Find the inverse of
 25 5 1
[A] =  64 8 1
144 12 1

Solution
From Example 4.6 in Chapter 4, we found
det ( A) = −84
Next we need to find the adjoint of [A] . The cofactors of A are found as follows.
The minor of entry a11 is
25 5 1
8 1
M 11 = 64 8 1 = = −4
12 1
144 12 1
The cofactors of entry a11 is
C11 = (− 1) M 11 = M 11 = −4
1+1

The minor of entry a12 is

25 5 1
64 1
M 12 = 64 8 1 = = −80
144 1
144 12 1
The cofactor of entry a12 is
C12 = (− 1)
1+ 2
M 12 = − M 12 = −(−80) = 80

Similarly
C13 = −384
C 21 = 7
 C 22 = −119
C 23 = 420
C 31 = −3
C 32 = 39
C 33 = −120
Hence the matrix of cofactors of [ A] is
− 4 80 − 384
[C ] =  7 − 119 420 


 − 3 39 − 120 
The adjoint of matrix [ A] is [C ]T ,
adj ( A) = [C ]
T

 −4 7 −3 

= 80 − 119 39 
 
 − 384 420 − 120
Hence
[A]−1 = 1
adj ( A)
det ( A)
 −4 7 −3 
1 
= 80 − 119 39 
− 84  
− 384 420 − 120
 0.04762 − 0.08333 0.03571 
=  − 0.9524 1.417 − 0.4643
 
 4.571 − 5.000 1.429 

If the inverse of a square matrix [A] exists, is it unique?

Yes, the inverse of a square matrix is unique, if it exists. The proof is as follows.
Assume that the inverse of [ A] is [B] and if this inverse is not unique, then let another
inverse of [ A] exist called [C ] .
If [B] is the inverse of [ A] , then
[ B][ A] = [ I ]
Multiply both sides by [C ] ,
[ B][ A][C ] = [ I ][C ]
[ B ][ A][C ] = [C ]
Since [C] is inverse of [ A] ,
[ A][C ] = [ I ]
Multiply both sides by [B] ,
[ B ][ I ] = [C ] , [ B ] = [C ]
This shows that [B] and [C ] are the same. So the inverse of [A] is unique.

Row reduction
Main article: Gaussian elimination

In row reduction, the linear system is represented as an augmented matrix:

This matrix is then modified using elementary row operations until it reaches reduced row
echelon form. There are three types of elementary row operations:

Type 1: Swap the positions of two rows.

Type 2: Multiply a row by a nonzero scalar.
Type 3: Add to one row a scalar multiple of another.

Because these operations are reversible, the augmented matrix produced always represents a
linear system that is equivalent to the original.

There are several specific algorithms to row-reduce an augmented matrix, the simplest of which
are Gaussian elimination and Gauss-Jordan elimination. The following computation shows
Gaussian elimination applied to the matrix above:
The last matrix is in reduced row echelon form, and represents the system x = −15, y = 8, z = 2. A
comparison with the example in the previous section on the algebraic elimination of variables
shows that these two methods are in fact the same; the difference lies in how the computations
are written down.

Cramer's rule
Cramer's rule is an explicit formula for the solution of a system of linear equations, with each
variable given by a quotient of two determinants. For example, the solution to the system

is given by

For each variable, the denominator is the determinant of the matrix of coefficients, while the
numerator is the determinant of a matrix in which one column has been replaced by the vector of
constant terms.

Though Cramer's rule is important theoretically, it has little practical value for large matrices,
since the computation of large determinants is somewhat cumbersome. (Indeed, large
determinants are most easily computed using row reduction.) Further, Cramer's rule has very
poor numerical properties, making it unsuitable for solving even small systems reliably, unless the
operations are performed in rational arithmetic with unbounded precision.
Homogeneous systems
A system of linear equations is homogeneous if all of the constant terms are zero:

A homogeneous system is equivalent to a matrix equation of the form

where A is an m × n matrix, x is a column vector with n entries, and 0 is the zero

vector with m entries.

Solution set
Every homogeneous system has at least one solution, known as the zero solution (or trivial
solution),

(OR)

A homogeneous system of linear equation is a linear system of equations where the right
hand sides of all the equations are zero. That is it is of the form

a11x1 + a12x2 + ... + a1nxn = 0

a21x1 + a22x2 + ... + a2nxn = 0
... ... ...
a m1x1 + am2x2 + ... + amnxn = 0

Notice that in matrix form this can be written as

Ax = 0

where A is the m x n matrix with entries aij, x the the n x 1 matrix with entries xi and 0 is
the n x 1 zero matrix. The augmented matrix's last column is the zero column. Since the
zero column is unaffected by elementary row operations, it is usually left out when the
computations are performed. The solution (0,0, ... , 0) is called the trivial solution. Any
other solution is called a nontrivial solution.
Theorem

Let

Ax = 0

be a homogeneous system with m x n matrix A. If m < n, then the system

always has a nontrivial solution.

This theorem just states that if there are more variables than equations, then there is a
nonzero solution.

Invertible matrix
In linear algebra, an n-by-n (square) matrix A is called invertible or nonsingular or
nondegenerate if there exists an n-by-n matrix B such that

where In denotes the n-by-n identity matrix and the multiplication used is ordinary matrix
multiplication. If this is the case, then the matrix B is uniquely determined by A and is
called the inverse of A, denoted by A−1. It follows from the theory of matrices that if

for square matrices A and B, then also

[1]

Non-square matrices (m-by-n matrices for which m ≠ n) do not have an inverse.

However, in some cases such a matrix may have a left inverse or right inverse. If A is m-
by-n and the rank of A is equal to n, then A has a left inverse: an n-by-m matrix B such
that BA = I. If A has rank m, then it has a right inverse: an n-by-m matrix B such that
AB = I.

While the most common case is that of matrices over the real or complex numbers, all
these definitions can be given for matrices over any commutative ring.

A square matrix that is not invertible is called singular or degenerate. A square matrix is
singular if and only if its determinant is 0. Singular matrices are rare in the sense that if
you pick a random square matrix, it will almost surely not be singular.
Matrix inversion is the process of finding the matrix B that satisfies the prior equation
for a given invertible matrix A.

Writing the transpose of the matrix of cofactors, known as an adjugate matrix, can also be
an efficient way to calculate the inverse of small matrices, but this recursive method is
inefficient for large matrices. To determine the inverse, we calculate a matrix of
cofactors:

where |A| is the determinant of A, Cij is the matrix of cofactors, and AT represents the
matrix transpose.

For most practical applications, it is not necessary to invert a matrix to solve a system of
linear equations; however, for a unique solution, it is necessary that the matrix involved
be invertible.

Inversion of 2×2 matrices

The cofactor equation listed above yields the following result for 2×2 matrices. Inversion
of these matrices can be done easily as follows: [2]

This is possible because 1/(ad-bc) is the reciprocal of the determinant of the matrix in
question, and the same strategy could be used for other matrix sizes.
Cramer's rule
Consider the linear system

which in matrix format is

Then, x and y can be found with Cramer's rule as:

and

The rules for 3×3 are similar. Given:

which in matrix format is

x, y and z can be found as follows:

Finding solution of a system of linear equations in the general

case
Consider a system of linear equations represented in matrix multiplication form as

where the square matrix A is invertible and the vector is the

column vector of the variables.

Then the theorem states that:

where Ai is the matrix formed by replacing the ith column of A by the column vector b.

This formula is, however, of limited practical value for larger matrices, as there are other
more efficient ways of solving systems of linear equations, such as by Gauss elimination
or, even better, LU decomposition.

Finding inverse matrix

Let A be an n×n matrix. Then

where Adj(A) denotes the adjugate matrix of A, det(A) is the determinant, and I is the
identity matrix. If det(A) is invertible in R, then the inverse matrix of A is
If R is a field (such as the field of real numbers), then this gives a formula for the inverse
of A, provided det(A) ≠ 0. The formula is, however, of limited practical value for large
matrices, as there are other more efficient ways of generating the inverse matrix, such as
by Gauss–Jordan elimination.

LU decomposition
In linear algebra, the LU decomposition is a matrix decomposition which writes a matrix as the
product of a lower triangular matrix and an upper triangular matrix. The product sometimes
includes a permutation matrix as well. This decomposition is used in numerical analysis to solve
systems of linear equations or calculate the determinant.

Definitions
Let A be a square matrix. An LU decomposition is a decomposition of the form

where L and U are lower and upper triangular matrices (of the same size), respectively.
This means that L has only zeros above the diagonal and U has only zeros below the
diagonal. For a matrix, this becomes:

Solving linear equations

Given a matrix equation

we want to solve the equation for a given A and b. In this case the solution is done in two
logical steps:

1. First, we solve the equation Ly = b for y

2. Second, we solve the equation Ux = y for x.
Note that in both cases we have triangular matrices (lower and upper) which can be solved
directly using forward and backward substitution without using the Gaussian elimination process
(however we need this process or equivalent to compute

the LU decomposition itself). Thus theLU decomposition is computationally efficient only when we
have to solve a matrix equation multiple times for different b; it is faster in this case to do an LU
decomposition of the matrix A once and then solve the triangular matrices for the different b, than
to use Gaussian elimination each time.

Inverse matrix
The matrices L and U can be used to calculate the matrix inverse by:

[citation needed]
Computer implementations that invert matrices sometimes use this approach.

Small example

One way of finding the LU decomposition of this simple matrix would be to simply solve the
linear equations by inspection. You know that:

Such a system of equations is underdetermined. In this case any two non-zero elements
of L and U matrices are parameters of the solution and can be set arbitrarily to any non-zero
value. Therefore to find the unique LU decomposition, it is necessary to put some restriction
on L and Umatrices. For example, we can require the lower triangular matrix L to be a unit one
(i.e. set all the entries of its main diagonal to ones). Then the system of equations has the
following solution:

l21 = 1.5
u11 = 4
u12 = 3
u22 = − 1.5.
Substituting these values into the LU decomposition above:

1.5. Computation of the inverse matrix

If
u11 u12 ··· u1n
u21 u22 ··· u2n
A-1 = : : : =( 1 2 ..., n)

un1 un2 ··· unn

is the inverse matrix of A, then (AA - 1 = I => )

0
:
a11 a12 ··· a1n u1i
0
a21 a22 ··· a2n u2i
1
(1) : : : : =
0
an1 an2 ··· ann uni
:
0

<=>

(1)' A i = i
so that i is determined by solving the linear system of equations (1), for example, by
applying the Gaussian elimination to the matrix
a11 ... a1n 0
: : 0
ai1 ... ain 1
= (A | i )
: : 0
an1 ... ann 0
and the inverse matrix is obtained by solving all n equations (1)', i = 1, ..., n. They all
have the same coefficient matrix A so that they can all be solved at the same time by the
Gaussian elimination.

a11 ... a1n 1 0 0 ... 0

a21 ... a2n 0 1 0 ... 0
: : : = (A | I ).
an1 ... ann 0 0 ... 1

If it is possible to convert (A | I) into the form

1 0 ··· 0 b11 b12 ··· b1n

0 1 0 ··· 0 b21 b22 ··· b2n
: : : : = (I | B)

0 ··· 1 bn1 bn2 ··· bnn

by means of the elementary row operations, then B = A - 1.

This is so, because (I | B) represents the n linear systems of equations, where the ith
system is

1· u1i + 0· u2i + ··· + 0uni = b1i

0· u1i + 1· u2i + 0 ··· + 0uni = b2i
:
0 + ··· + 0 + 1· uni = bni

<=>

b1i
b2i
i =
:
bni
1.6. LU decomposition
Assume that a square matrix A can be written as a product

(1)

LU = A,

where L is a lower triangular matrix and U is an upper triangular matrix;

l11 0 0 0 u11 u12 u13 u14 a11 a12 a13 a14

l21 l 22 0 0 0 u22 u23 u24 a21 a22 a23 a24
l31 l 32 l33 0 0 0 u33 u34 = a31 a32 a33 a34 .

l41 l42 l43 l44 0 0 0 u44 a41 a42 a43 a44

Then the linear system of equations A = , i.e.,

(2)

A = (LU) = L (U ) = ,

can be solved by first solving the lower triangular system

(3)

L =

and then the upper triangular system

(4)

U = .

The equation (3) is solved by the method of forward substitution,

(5)

y1 = b1 / l11 ,

yi
= (1 / lii ) [ bi - lij yj ] , i = 2, 3, ..., n,

while the equation (4) is solved using the backward substitution,

(6)

xn = yn / unn ,

xi
= (1 / uii ) [ yi - uij xj ] , i = n - 1, n - 2, ..., 1.

Note. LU decomposition is not unique: if A = LU, then A = LDD - 1U = (LD) (D - 1U) =

L1 U1 is again an LU decomposition, if D is a diagonal matrix. An additional assumption
lii = 1 i = 1, ..., n, guarantees the uniqueness.

The construction of an LU decomposition can be done by the Crout's algorithm, for

example.

Usually we construct an LU decomposition by means of the Gaussian elimination in such

a way that L is formed from the coefficients used in the elimination while U is the matrix
A itself after the elimination.
Find the LU decomposition of
936
461
A=
117

Solution:
We use Gaussian elimination on A, so that the resulting upper triangular matrix will be U
and the lower triangular matrix which is formed from the opposite numbers of the
coefficients used will be L.
936 93 6 93 6
461 0 14/3 -5/3 0 14/3 -5/3
~ ~
117 0 2/3 19/3 00 46/7

-Add the 1st row multiplied

by -4/9 to the 2nd row
=>the 1st entry on the 2nd
row of L is -(-4/9) = 4/9
-Add the 1st row multiplied
by -1/9 to the 3rd row
=>the 1st entry on the 3rd row
of L is -(-1/9)=1/9
-Add the 2nd row multiplied
-We have the upper
by -1/7 to the 3rd row
triangular matrix U
=>the 2nd entry on the 3rd
row of L is -(-1/7) = 1/7

Moreover,
1 0 0
4/9 1 0
L=
1/9 1/7 1

so that the LU decomposition is

936 1 0 0 93 6
461 4/9 1 0 0 14/3 -5/3 =
A= =
LU
117 1/9 1/7 1 00 46/7

The result can be checked by multiplying L and U.

Crout matrix decomposition
In linear algebra, the Crout matrix decomposition is an LU decomposition which
decomposes a matrix into a lower triangular matrix (L), an upper triangular matrix (U)
and, although not always needed, a permutation matrix (P).

The Crout matrix decomposition algorithm differs slightly from the Doolittle method.
Doolittle's method returns a unit lower triangular matrix and an upper triangular matrix,
while the Crout method returns a lower triangular matrix and a unit upper triangular
matrix.

3.4. The determinant of a matrix

The determinant of a 1 x 1 matrix A = (a11 ) , det A, is a11

The determinant of an n x n matrix

a11 a12 ... a1n

a21 a22 ... a2n
A=
:
an1 an2 ... ann

a11 ... a1n

a21 ... a2n
is det A = a11 A11 + a12 A12 + ··· + a1n A1n = ,
:
an1 ... ann

where A1j is (-1)1+j times the determinant of the (n - 1) x (n - 1) matrix, which is obtained
from A by deleting the first row and the jth column. For example, the determinant of a 2 x
2 matrix is

a11 a12
a21 a22 = a11 (-1)1+1 a22 + a12 (-1)1+2a21 = a11 a22 - a12 a21
and the determinant of a 3 x 3 matrix is

a11 a12 a13

a21 a22 a23 a22 a23 a21 a23 a21 a22
= a11 - a12 + a13 .
a32 a33 a31 a33 a31 a32
a31 a32 a33

Example 1: Calculation of a determinant

It can be shown that det A = ai1 Ai1 + ai2 Ai2 + ··· + ain Ain
,
= a1j A1j + a2j A2j + ··· + anj Anj

where A1j is (-1)1+j times the determinant of the (n - 1) x (n - 1) matrix, which is obtained
from A by deleting the ith row and the jth column.

The number Aij is called the cofactor of the element aij .

It follws from the definition that

1) if A has a 0 row or a 0 column, then det A = 0.

2) det AT = det A.

3) if a row or a column of A is multiplied by k, the determinant of the matrix obtained =

kdet A => det (kA) = kndet A.

4) if any two rows (or columns) of A are interchanged, the determinant of the matrix
obtained = -det A.

5) if any two rows (or columns) of A are the same, then det A = 0.

6) If c x (kth row) is added to the ith row, i k, the determinant of the matrix obtained =
det A. The same rule applies to columns.
7)

a11 + x1 a12 ··· a1n a11 a12 ··· a1n x1 a12 ··· a1n
a21 + x2 a22 ··· a2n a21 a22 ··· a2n x2 a22 ··· a2n
= +
: : : : :
an1 + xn an2 ··· ann an1 an2 ··· ann xn an2 ··· ann

The same rule applies to rows.

8) The determinant of an upper triangular matrix is the product of the diagonal elements.

Example 2: The determinant of an upper triangular matrix

9) det AB = det A· det B.

Example 3: The determinant of a product

The adjugate matrix of A is

A11 A21 ··· An1

A12 A22 ··· An2
adj A = : : = (Aij )T

A1n A2n ··· Ann

Proposition 8. A adj A = adj A · A = det A · I

Proof. Denote A adj A = (bij )

=> bij = ai1 Aj1 + ai2 Aj2 + ··· + ain Ajn

=>

1) If i j then bij is the determinant of the matrix obtained from A by replacing the jth row
by the ith row. => two equal rows => det = 0 = bij .

2) If i = j, then bij = the determinant of A computed across the ith row.

 det A 0 ··· 0
0 det A 0 ··· 0
=> A adj A = = det A· I
: :
0 ··· 0 det A

Similarly adj A · A = det A· I.

Peoposition 9.

1) A has an inverse <=> det A 0

2) If det A 0, then

A -1 = (1 / det A) adj A.

Proof. 2) If det A 0, then (1 / det A) adj A · A = I => A -1 = (1 / det A) adj A

1) If A has an inverse, then I = AA - 1 => 1 = det I = det (AA -1) = det A ·det A -1 => det A
0.

Note. det A 0 <=> A -1 exists <=> the rank of A is n.

If det A = 0, then the rank of A < n and thus the nullity of A = n - (the rank of A) 1
=> The equation A = 0 has a solution 0.

Applications of determinant
Everything in this section is based on the following fact:

A homogeneous linear system of n equations in n unknowns has nontrivial solutions

if and only if the determinant of the coefficient matrix is zero.

A =0

1. If det A 0 => A-1 exists <=> A-1 A = A-1 = 0 <=> =

2. If det A = 0, then the rank of A is not full; it is n - r < n and there are n - r
unknowns which can be chosen arbitrarily.
=> the system has nontrivial ( 0) solutions.
Example 1: Calculation of a determinant

Calculate the determinant of the matrix

1200
0102
A= 0230 .

3010

Solution:

1200 102 002 012 010

0102 230 030 020 023
det A = = (-1)1+1·1 + (-1)1+2·2 +0 -0
0230
010 310 300 301
3010

-Compute the determinant across the first row and obtain four 3-rowed determinants
-Compute the first one of the 3-rowed determinants across the 3rd row(this row has two
zeros) and the second one across the first row
-The two last of the 3-rowed determinants vanish
-Remember that we could equally well compute across columns too
-Each subdeterminant has a coefficient of the form (-1)i+jaij