Monica Martinez-Bravo Last Updated: Feb 20th, 2011 Calculus with Matrix Algebra This handout will clarify

how to take derivatives with respect to vectors. The proofs are beyond the scope of the course, but this handout might still be helpful in illustrating how to make basic operations with matrices and vectors.

A scalar-based function y = f (x1 ; x2 ; :::; xk ) can be expressed in vector form by y = f (x) where bold denotes a vector. x has dimesions (k y = a0 x where a is a (k 1) vector. 6 7 i6 x 7 6 2 7 7 = a1 x1 + a2 x2 + ::: + ak xk ::: ak 6 6 ::: 7 4 5 xk 2 x1 3 1). A linear function y = a1 x1 + a2 x2 + ::: + ak xk can also be expressed in matrix algebra form by

y=

a1 a2

Notice that with linear functions we have y = a0 x = x0 a. Multiply the matrices in the 2nd way to convince yourself. The vector of partial derivatives of function f (x) with respect to vector x is dened by 2
@y @x1 @y @x2

In the linear vector function case it is easy to prove that 6 7 6 6 7 6 6 7 6 =6 7=6 6 ::: 7 6 4 5 4
@y @xk

6 7 7 @f (x) 6 6 7 =6 7 6 ::: 7 @x 4 5
@y @xk

@f (x) @x

=a 2 a1 3

@f (x) @x

@y @x1 @y @x2

@(a1 x1 +a2 x2 +:::+ak xk ) @x1 @(a1 x1 +a2 x2 +:::+ak xk ) @x2

:::
@(a1 x1 +a2 x2 +:::+ak xk ) @xk

7 7 6 7 6 a 7 7 6 2 7 7=a 7=6 7 6 ::: 7 5 4 5 ak

Property 1. Hence, the property that we have just proved is that @a0 x =a @x 1

when a and x are (k

1) vectors.

Let us now check whether this applies for a system of linear equations. Consider matrix A of dimensions (4 3), and vector x is (3 1). Now we have a system of linear regressions: y = Ax Notice that y is now bold because is also a vector (4 of linear equations: 2 3 2 3 1). That why we say we have a system s 2 3 7 7 7 7 7 5

7 6 6 6 y 7 6 a 6 2 7 6 21 a22 a23 y=6 7=6 6 y3 7 6 a31 a32 a33 5 4 4 a41 a42 a43 y4

y1

a11 a12 a13

6 7 x1 76 7 6 a21 x1 + a22 x2 + a23 x3 6 76 7 4 x2 7 = 6 5 6 a x +a x +a x 7 32 2 33 3 4 31 1 5 x 3 a41 x1 + a42 x2 + a43 x3

a11 x1 + a12 x2 + a13 x3

What does it mean to take derivatives of y with respect to x? Notice that they are both vectors. In other words y = Ax is actually a system of equations. Finding the rst order derivatives is actually taking the derivative of each equation, with respect to each variable. Hence, the result is necessarily a matrix. 2
@y1 @x1 @y1 @x2 @y1 @x3 @y2 @x1 @y2 @x2 @y2 @x3 @y3 @x1 @y3 @x2 @y3 @x3 @y4 @x1 @y4 @x2 @y4 @x3

@y @x

@y1 @x

@y2 @x

@y3 @x

@y4 @x

6 =6 4

7 6 7 7 = 6 a12 a22 a32 a42 7 = A0 5 4 5 a13 a23 a33 a43

a11 a21 a31 a41

where the rst equality follows by denition and the last two equalities are the application to our linear case. Property 2. Hence, this proofs the second property @Ax = A0 @x where A is a matrix (h k), and x is a vector (k 1).

Now consider the following quadratic form: q = x0 Ax where A is a symmetric matrix (k quadratic form). Then, it can be shown that: k), and x is a vector (k 1). (This is the denition of a

Property 3.

@x0 Ax = 2Ax @x k), and x is a vector (k 1).

where A is a matrix (k

For further information I attach some derivations that you can nd in "William H. Greene (2005) Econometric Analysis. Pearson Education, New York NY."