Autocorrelation Function

Sample Autocorrelation function

Partial Autocorrelation Function

Chapter 6 Model Specification

STTN315

Department of Statistics, North-West University, Potchefstroom, South Africa

Leonard.Santana@nwu.ac.za

May 26, 2016

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function
Partial Autocorrelation Function

Contents

Properties of the Sample Autocorrelation Function

The Partial and Extended Autocorrelation Function
Specification of Some Simulated Time Series
Nonstationarity
Othe Specification Methods
Specification of Some Actual Time Series

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function
Partial Autocorrelation Function

Introduction

This chapter deals with model specification/identification.

Intererested in finding tentative values for p, d, and q.
(Next chapter deals with estimating the s, s, and e s
the following chapter looks at checking its adequacy).
The procedure we follow was popularised by G.E.P. Box and
G.M. Jenkins (1976) and is often called the Box-Jenkins
method.

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function Patterns
Partial Autocorrelation Function

Example of patterns: ACF

Well be looking at the ACF to identify patterns. What should

we see though?

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function Patterns
Partial Autocorrelation Function

AR(1)
For the AR(1) model, recall that
k = k1 , k = 0, 1, 2, . . .

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function Patterns
Partial Autocorrelation Function

AR(2)
For the AR(2) model, recall that
   k
1 + 2 1
k = 1 + k , k = 0, 1, 2, . . .
1 2 2

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function Patterns
Partial Autocorrelation Function

MA(q)

For the MA(q) model, recall that

k + 1 k+1 + 2 k+2 + + qk q


for k = 1, 2, . . . , q
1 + 12 + + q2

k =


0 for k > q

i.e., it will have values different from zero only at lags

k = 1, 2, . . . , q. The rest are zero.
Conclusion: The ACF is going to be a GREAT indicator
of whether we have an MA(q) process.

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Sample Autocorrelation function Patterns
Partial Autocorrelation Function

ARMA(1,1)

For the ARMA(1,1) model, recall that

(1 1 1 )(1 1 ) k1
k = 1 , k = 1, 2, . . .
1 21 1 + 12

This will also tail off slowly as k increases.

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Sample Autocorrelation function

As you know, we will be making use of the Sample

Autocorrelation function (sample ACF) to identify the
appropriate model.
The Sample ACF is defined as
Pn
(Yt Y )(Ytk Y )
rk = t=k+1 Pn 2
t=1 (Yt Y )

We want to identify patterns in rk that are characteristic of

the known patterns in k for common ARMA models.

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Properties of rk : Large Sample Theory

The sampling properties of rk are difficult to obtain (even the

expected value is tough).
We will focus on Large Sample results. And be happy with it.
To start the discussion, first assume that the series is
stationary,
follows some kind of ARMA model, and
given by

X
Yt = + j etj
j=0
P
with the stationarity consditions j=0 |j | < and
P 2
j=0 jj < .

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Properties of rk : Large Sample Theory

Under these conditions, the variables

n(r1 1 ), n(r2 2 ), . . . , n(rm m )

follow a joint normal distribution with

zero mean,
variance cjj , and
covariance cij ,
where

X
cij = (k+i k+j +ki k+j 2i k k+j 2j k k+1 +2i j 2k )
k=

as n . (i.e., this is only a large sample result and not

true in general).
STTN315 STTN315 Chapter 6
 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Properties of rk : Large Sample Theory

That was a mouthful.

A more palatable answer is:

 ckk 
rk N k , .
n
and
ckj
Corr (rk , rj )
ckk cjj
as n .
P
Note that cii = k= (k+i + ki 2i k )2 .
But what does this mean for us?

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Special case: White Noise

Suppose that {Yt } is White Noise.

Using these formulas we find that
1
Var(rk )
n
and
Corr(rk , rj ) 0, k 6= j.
DIY proof

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Special case: AR(1)

Suppose that {Yt } is AR(1).

Using these formulas with i = j we find that

1 (1 + 21 )(1 2k
 
1 ) 2k
Var(rk ) 2k1
n 1 21

and, if k is very large, we have

1 (1 + 21 )
 
Var(rk )
n 1 21

What does this mean?

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Special case: MA(1)

Suppose that {Yt } is MA(1).
Using these formulas we find that
c11 = 1 321 + 441 ,
ckk = 1 + 221 , for k > 1
and
c12 = 21 (1 21 )
The variance is then
1
1 321 + 441

Var(r1 )
n
and
1
1 + 221

Var(rk )
n
STTN315 STTN315 Chapter 6
 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

Special case: MA(q)

Suppose that {Yt } is MA(q).

Using these formulas we find that for i = j = k
q
X
ckk = 1 + 2 2j , for k > q
j=1

The variance is then and

q
1 X
Var(rk ) 1+2 2j
n
j=1

But what does this mean for us?

STTN315 STTN315 Chapter 6

 Autocorrelation Function
Large Sample Properties
Sample Autocorrelation function
Special cases
Partial Autocorrelation Function

What can we do with these expressions?

Since we know that rk is approximately normally distributed

with mean k and variance ckk /n, this can be used to perform
tests of the form

H0 : k = 0 vs. HA : k 6= 0

How?
First we need to estimate these variance expressions though.
How would we do that?

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Correlation

What is partial correlation between X and Y , controlling for

Z?
The correlation between X and Y , with the effect of Z
removed.
It is defined in this simple case as
XY XZ YZ
XY Z = q q
1 2XZ 1 2YZ

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial AUTOcorrelation Function

The ACF is good for identifying MA(q) models, but what

about AR(p) models?
Well use the Partial Autocorrelation Function (PACF) for this.
The PACF is defined as the correlation between Yt and Ytk
after removing the effect of the intervening variables
Yt1 , Yt2 , Yt3 , . . . Ytk+1 .

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function

We will denote partial autocorrelation at lag k by kk

If {Yt } is a normally distributed time series, we can write it in
the following special (simple) form

kk = Corr (Yt , Ytk |Yt1 , Yt2 , Yt3 , . . . Ytk+1 )

That is, kk is the (partial) correlation between Yt and Ytk

conditional on Yt1 , Yt2 , . . . , Ytk+1 .
STTN315 STTN315 Chapter 6
 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function

A non-normal (alternative) approach:

Consider predicting Yt using a linear function of the
intervening variables Yt1 , Yt2 , , Yt(k1) , say,

Ybt = 1 Yt1 + 2 Yt2 + + k1 Yt(k1) ,

The s are chosen to minimize the mean square error of

prediction,

E((Yt Ybt )2 ) = E((Yt 1 Yt1 2 Yt2 k1 Yt(k1) )2 ).

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function

From stationarity, it follows that the best predictor of Ytk

based on the same intervening variables Yt1 , Yt2 , ,
Yt(k1) will also be

Ybtk = 1 Yt1 + 2 Yt2 + + k1 Yt(k1) ,

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function

The partial autocorrelation function at lag k is then defined to

be the correlation between the prediction errors; i.e., the
correlation
 
Cov (Yt Ybt )(Ytk Ybtk )
kk = q q
Var(Yt Ybt ) Var(Ytk Ybtk )

Interpret this as the correlation between Yt and Ytk with

the intervening bits removed.

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function

With a bit of math (OK, a LOT of math) we can show that

Var(Yt Ybt ) = Var(Ytk Ybtk ) = 0 1 1 k1 k1

and
 
Cov (Yt Ybt )(Ytk Ybtk ) = k 1 k1 k1 1

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function

Therefore
k 1 k1 k1 1
kk =
0 1 1 k1 k1
0
Multiplying by 0 we get

k 1 k1 k1 1
kk =
1 1 1 k1 k1

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function : k = 2

Consider now a simple example where k = 2.
It can be shown in this case that best linear predictor is
Ybt = Ybt2 = 1 Yt1 .
That is, 1 = 1 .
The partial correlation in this case is then
2 1 1 2 21
22 = =
1 1 1 1 21

Note: If we have an AR(1) process, then k = k1 , and then

2 21 21 21
22 = = = 0.
1 21 1 21
This holds for all k > 1 in an AR(1) process.
STTN315 STTN315 Chapter 6
 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function: AR processes

Consider a general AR(p) case.
In Chapter 9 it is shown that the best linear predictor of Yt is
Ybt = 1 Yt1 + 2 Yt2 + + p Ytp ,
when we base it on the sequence
Yt1 , Yt2 , , Ytp , , Yt(k1)
i.e., k > p.

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function: AR processes

Let the best linear predictor of Ytk be some linear function,

say,

Ybtk = 1 Yt1 + 2 Yt2 + + tp Ytp + + k1 Yt(k1)

Then, for a general AR(p) process, we have

Cov(Yt Ybt , Ytk Ybtk )

= Cov(Yt 1 Yt1 2 Yt2 p Ytp ,
Ytk 1 Yt1 2 Yt2 tp Ytp
k1 Yt(k1) )
= Cov(et , Ytk 1 Yt1 2 Yt2 tp Ytp
k1 Yt(k1) ) = 0

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Partial Autocorrelation Function: MA processes

Note that for a MA(1) process, the 2nd lag PACF is given by

1k (1 12 )
kk = 2(k+1)
, k 1.
1 1

Note what happens as k increases.

STTN315 STTN315 Chapter 6

 Autocorrelation Function Basic partial correlation (not auto)
Sample Autocorrelation function Partial Autocorrelation Function
Partial Autocorrelation Function Sample Partial Autocorrelation Function

Sample Partial Autocorrelation Function

By replacing the autocorrelation (i ) terms with sample

autocorrelations (ri ) we obtain the sample PACF.

STTN315 STTN315 Chapter 6