University of California, Los Angeles

Department of Statistics
Statistics 100B

Instructor: Nicolas Christou

Moment generating functions

Definition:
MX (t) = EetX
Therefore,
If X is discrete
X

MX (t) =

etX P (x)

If X is continuous
Z

MX (t) =

etX f (x)dx

Aside:
ex = 1 +

x2 x3
x
+
+
+
1!
2!
3!

Similarly,
tx (tx)2 (tx)3
+
+
+
1!
2!
3!
Let X be a discrete random variable.
etx = 1 +

tx (tx)2 (tx)3
1+
+
+
+ P (x)
1!
2!
3!

"

MX (t) =

tX

e P (x) =

X
x

or
MX (t) =

P (x) +

t X
t2 X 2
t3 X 3
xP (x) +
x P (x) +
x P (x) +
1! x
2! x
3! x

To find the kth moment simply evaluate the kth derivative of the MX (t) at t = 0.
th derivative
EX k = [MX (t)]kt=0

For example:
First moment:
MX (t)0 =

xP (x) +

We see that MX (0)0 =

2t X 2
x P (x) +
2! x

xP (x) = E(X).

Similarly,
Second moment
MX (t)00 =

x2 P (x) +

We see that MX (0)00 =

Examples:
Find the moment
Find the moment
Find the moment
Find the moment

6t X 3
x P (x) +
3! x

x2 P (x) = E(X 2 ).

generating
generating
generating
generating

function
function
function
function

of
of
of
of

X b(n, p).
X P oisson().
X exp().
Z N (0, 1).

Theorem:
Let X, Y be independent random variables with moment generating functions MX (t), MY (t)
respectively. Then, the moment generating function of the sum of these two random variables
is equal to the product of the individual moment generating functions:
MX+Y (t) = MX (t)MY (t))
Proof:

Let X, Y independent random variables.

Use this theorem to find the distribution of X + Y , where X b(n1 , p), Y b(n2 , p).
Use this theorem to find the distribution of X+Y , where X P oisson(1 ), Y P oisson(2 ).
Use this theorem to find the distribution of X + Y , where X N (1 , 1 ), Y N (2 , 2 ).

Properties of moment generating functions:

Let X be a random variable with moment generating function MX (t) = EetX , and a, b are
constants
1. MX+a (t) = eat MX (t)
2. MbX (t) = MX (bt)
a

3. M X+a = e b t MX ( bt )
b

Proof:

Use these properties and the moment generating function of Z N (0, 1) to find the moment
generating function of X N (, )

Let X1 , X2 , . . . , Xn be i.i.d. random variables from N (, ). Use moment generating functions to find the distribution of
a. T = X1 + X2 + . . . + Xn .

=
b. X

Pn
i=1

Xi

Distribution of the sample mean - Sampling from normal distribution

follows exactly the normal distribution with
If we sample from normal distribution N (, ) then X

mean and standard deviation n regardless of the sample size n. In the next figure we see the
The first figure is the N (5, 2) distrieffect of the sample size on the shape of the distribution of X.

bution. The second figure represents the distribution of X when n = 4. The third figure represents
when n = 16.
the distribution of X

0.4
0.0

0.2

f(x)

0.6

0.8

N(5,, 2)

2
4

0.4
0.0

0.2

f(x)

0.6

0.8

N(5,,

N(5,,

0.4
0.2
0.0

f(x)

0.6

0.8

16

5
x

11

13