You are on page 1of 3

Matrix Di erentiation

Andrew Fitzgibbon, Naoufel Werghi


August 28, 1997

1 Introduction
This is a collection of di erentiation rules for matrices and vectors. Uppercase letters are matrices,
lowercase are vectors.

2 Dierentiation with respect to a vector


The rst table is for derivatives with respect to a column vector x. Note that x xT is then a 3 3
matrix.
Form Derivative wrt x.
kxk2 2x
kg(x)k2 2rxg(x)g(x)
kx + ak2 2(x + a)
kAxk2 2AT Ax
kAx ; bk2 2AT (Ax ; b)
Ax AT
x Ax (AT + A)x
T
a  x a0 1
;a3 a2 0
a x @ a3 0 ;a1 A
;a2 a1 0
ka xk2 2(kak2I ; aaT )x
3 Scalar function of vector
Let's consider g(x) a scalar function of x and v(x) vectorial function of x
then the derivative of g(x)v(x) is v(g )T + gv
0 0

(') means the derivative with respect to x

4 Scalar functions of matrices


This next table is derivatives of some common scalar functions of a matrix A. The results are
matrices the same size as A, with the derivative dened as:
0 @f @f  @f 1
BB @a11 @a12 @a1n CC
df BB @f
@a21
@f
@a22  @f
@a2n CC
dA = B@ ..
.
..
.
... ..
.
CA
@f @f  @f
@an1 @an2 @ann
Form Derivative wrt A.
bT Ax bxT
kAxk2 2AxxT
kAx + bk2 2(Ax + b)xT
2A(xxT ) + 2bxT
kA ; AT k22 4(A ; AT )
kAk22 2A
det(A) A = det(A)A 1 = adj (A)
y ;

tr(A) I
xT AT MAx 2MAxxT
kAAT ; I k22 4(AAT ; I )A
Ax Some horrible tensor . . .
5 Those horrible tensors in full
OK, we want to di erentiate Ax. What do we mean? Well each of the 3 components of x has a
matrix of derivatives:
@xi
@apq
And that's our answer. The question is how to do anything with it. Well, rst let's write out Ax
in longhand: X
Ax]i = aij xj
j
And using the summation convention1 we can drop the sigma. Now we have a table translating
common matrix operations to the summation form.
Operation Summation Form
a = x  y ai = xk yk
y = Ax yi = Aik xk
C = AB Cij = Aik Bkj
tr(A) Akk
AT AT ]ij = Aji
Note that we often need to bring in a new index to \do" the summation for us. Also, if we want
to have say uT v + aT b, we must use a di erent index for each half: up vp + aq bq for example.
@ bT Ax, which is given above as bxT . First we write
Now let's look at a specic problem: Find @A
the expression in the summation convention:
bT Ax = b  (Ax)
= b  Aij xj ]3i=1
= bi Aij xj

Now we've got two summation signs on the outside, we can just bring in the derivative operator
and apply normal rules to
@ @Aij
@A (bi Aij xj ) = bi @A xj
pq pq
@A is zero unless i = p and j = q . This is written using the Kronecker delta
Finally we note that @A ij

as:
pq

@Aij
@A = ip jqpq
1 Any repeated subscript is summed over.
Which goes back into the original problem to give
 @bT Ax  @
@A pq = @Apq (bi Aij xj )
= bi ip jq xj
= bp xq
= bxT ]pq

You might also like