Normal and Gamma Distributions

Normal distribution

The probability density function of a normal distribution (or Gaussian distribution) is f (x ; , 2 ) = 1 2 exp{ (x )2 }, 2 2

which will be denoted by N (, 2 ), where is a location parameter and is a scale parameter. N (0, 1) is a standard normal distribution.

Expectation and variance

If Z N (0, 1), then Y = + Z N (, 2 ). If Y N (, 2 ), then Z =

Y

N (0, 1).

E (Z ) = 0 and Var(Z ) = 1. Then E (Y ) = and Var(Y ) = 2 .

Simulation of normal random variables: Polar coordinate method

Let X and Y be two independent standard normal distributed random variables. Consider the polar transformation: X = Y = 2R sin() 2R cos();

where 0 < < 2 and 0 < R < . Then R exponential(1) and Unif(0, 2 ). Therefore, we can generate R = log(U1 ) and = 2 U2 where U1 , U2 are two independent Unif[0, 1] random variables.

Normal density plots

0.4

mu=0,sigma=1 mu=1.5,sigma=1 mu=1.5,sigma=2 mu=1.5,sigma=3

Density

0.0 6

0.1

0.2

0.3

0 x

MGF and linear combination

The moment generating function of a standard normal distribution is MZ (t ) = exp(t 2 /2) and hence the MGF for X N (, 2 ) is MX (t ) = exp(t + 2 t 2 /2). Let X1 , , Xn be independent random variables with Xi N (i , i2 ) for i = 1, , n. Then
n n n

ci Xi N (
i =1 i =1

ci i ,
i =1

ci2 i2 ).

Example: Normal approximation to Binomial

Suppose X has a Binomial distribution with parameters n and p. Then X can be written as X = I1 + + In where Ii are independent Bernoulli(p). Then X is a sum of independent and identically distributed random variables. As we will see later, the distribution of X can be approximated by normal distribution if n is large and p is not extremely close to 0 and 1. If n = 25 and p = 0.6, then 13 15 . 13 15 X 15 ) = P (Z ) = 0.206, P (X 13) = P ( 6 6 6 where Z N (0, 1).

Lognormal distribution
If log(X ) N (, 2 ), then X is said to have lognormal distribution. The pdf of X is f (x ; , 2 ) = (log(x ) )2 1 exp{ } 2 2 2 x 1 0 < x < .

If X lognormal(, 2 ), then E (X ) = exp( + 2 /2) Var(X ) = exp(2( + 2 )) exp(2 + 2 ).

Lognormal density plots

mu=0,sigma=1 mu=1.5,sigma=1 mu=1.5,sigma=2 mu=1.5,sigma=3

Density

0.0 0

0.1

0.2

0.3

0.4

0.5

0.6

4 x

Gamma distribution
The Gamma distributed random variable T with parameters and 1 has the density function f (t ) = 1 1 t t e () 0 < t < .

which is an extension of Gamma function. The full Gamma distribution family is a scale transformation of T . Dene X = T . Then f (x ; , ) = 1 x 1 ex / () 0 < x < ,

where is a shape parameter and is a scale parameter. We will denote it as Gamma(, ).

Mean, variance and MGF

If X Gamma(, ), then E (X ) = and Var(X ) = 2 . The MGF of X is MX (t ) = 1 1 t

t < 1 .
n i =1 i , ).

If Xi Gamma(i , ), then

Xi Gamma(

Gamma density plots

1.5

alpha=1,beta=1 alpha=3,beta=1 alpha=2,beta=3 alpha=1,beta=3

Density 0.0 0 0.5

1.0

4 x

Connection with Chi-square distribution

If we set = p/2 and = 2, then the Gamma distribution has pdf f (x ; p ) = 1 x p/21 ex /2 0 < x < . (p/2)2p/2

We call it Chi-square distribution and denote it by 2 p. It is easy to see that E (X ) = p and Var(X ) = 2p if X having Chi-square distribution. If Z N (0, 1), then X = Z 2 2 1.

Gamma and Poisson relationship

Let D1 , D2 , be iid exponential(1) random variables. Dene X1 = D1 , X2 = X1 + D2 , X3 = X2 + D3 , . Assume that the number of occurrences Y in the time interval (0, ) has Poisson(). Then P (Xr > ) = P (Y r ).