Basic Time Series

Linear Time Series Analysis
Lecture 1: Some Basic Time Series Concepts
Daniel Buncic
Institute of Mathematics & Statistics
University of St. Gallen
Switzerland
December 12, 2013

Version: [ltsa1-a]
Homepage
www.danielbuncic.com
University of St. Gallen
Outline/Table of Contents
Outline
Introduction
Overview
Descriptive Analysis
Probabilistic Approach
Examples of Time Series
Objectives of Time Series Analysis
Basic Concepts
Some Definitions
Examples of Time Seriess
General Approach to Time Series Modeling
Stationary Models
Autocovariance and Autocorrelation
Some Model Based Examples
Sample Autocovariance and Autocorrelation
Estimation and Elimination of Both Trend and
Seasonality
Lag (or Backshift) Operator
Exercises
Daniel Buncic (University of St. Gallen)
Lecture 1: Linear Time Series Analysis
December 12, 2013
2/65
Introduction
Overview
Overview
A time series is a set of observations xt , each one being recorded at a specific time t:
discrete-time time series, when the set T0 of times at which observations are
made is a discrete set;
continuous-time time series, when observations are recorded continuously over

some time interval.
Of particular interest are discrete-time time series with observations recorded at fixed
time intervals. Time-distance between observations is called frequency.
Typical frequencies used in practice: daily, monthly, quarterly, . . .
December 12, 2013
3/65
Introduction
Plot a two-dimensional graph of recording times t (X-axis) vs. observations
yt , t = 1, . . . , T (Y -axis).
Generally, some smoothing techniques are applied:
rolling means: instead of yt plot

1
yt d1 + . . . + yt + . . . + yt+ d1
yt =
2
2
d
(1)
where d is an odd positive integer.
exponential smoothing: instead of yt plot
yt = ayt + (1 a)yt1
(2)
with y1 = y1 and 0 < a < 1.
December 12, 2013
4/65
Introduction
Preliminary goal of this analysis: examine the main features of the graph and check
whether there is:
a) a trend component (linear, quadratic, . . .);
b) a seasonal component;
c) a cyclical component;
d) any apparent sharp changes in behavior;
e) any outlying observations.
December 12, 2013
5/65
Introduction
The observed time series {yt }tZ is a realization or sample of an underlying unknown
stochastic process {Yt }tZ .
Final goal of the analysis:
explore the main characteristics of the underlying stochastic process, given the
information included in the observed sample.
generally, analysis is performed on stationary time series. If the time series is

not stationary, then some transformations of the data are done to reach
stationarity.
note that almost all modern economic time-series are not stationary and when
transforming the data some important information on the original stochastic
process can be lost.
December 12, 2013
6/65
Introduction
Possible solution:
if it exists, take a linear combination of two (or more) stochastic processes

that are non-stationary.
if this linear combination returns a series that is stationary, then the two series
have a common stochastic trend or permanent component
the series are then said to be cointegrated
least-squres is valied for fitting but adjustments to standard errors need to be

made when using statistical tests
December 12, 2013
7/65

Figure 1: Monthly sales in kiloliters of red wine by Australian winemakers from January 1980
through October 1991 (142 recorded times, frequency: monthly)
upward trend and seasonal pattern with peaks in July and troughs in January.
December 12, 2013
8/65

Figure 2: Accidental deaths in the US. Monthly accidental deaths in the US from January
1973 through December 1978 (72 observations, frequency: monthly).
strong seasonal pattern, with maximum for each year occurring in July and
minimum in February
not apparent trend
December 12, 2013
9/65

Figure 3: Population in the US. Population of the US measured at ten-year intervals from
1790 to 1990 (21 observations, frequency: 10 years).
upward trend is evident, can fit a quadratic or exponential trend to the data.
December 12, 2013
10/65

Figure 4: A signal extraction problem. 200 observations from the process
Xt = cos(t/10) + Nt , t = 1, . . . , 200, are simulated, where Nt independent,
Nt N (0, 0.25). Such a series is often referred to as signal plus noise model.
need to determine the unknown signal component smooth the data by

expressing Xt as a sum of sine waves of various frequencies
eliminate high-frequency components spectral analysis.
December 12, 2013
11/65

Figure 5: Two macroeconomic factors in the US. US monthly Help Wanted Advertising in
Newspapers (HELP) Index and US Industrial Production Index (IP) observations from
January 1960 to December 2001 (504 observations, frequency: monthly).
HELP Index: random fluctuation around a slowly changing level.
IP: evident upward trend, yearly seasonality component.
December 12, 2013
12/65

Figure 6: Daily US S&P500 Index values and log-returns for the period between January, 1st
2003 to December, 30th 2005 (783 observations, frequency: daily).
upward trend in the Index values is evident,
log-returns show that one can eliminate trend in Index by differencing.
heteroskedasticity in the return series need model for volatility
December 12, 2013
13/65

Figure 7: Daily Swiss Credit Suisse share values and log-returns for the period between
January, 1st 2003 to December, 30th 2005 (783 observations, frequency: daily).
as above for the index.
December 12, 2013
14/65
the final goal of time series analysis is to introduce some techniques for
drawing inferences from time series yt , t = 1, . . . , T of realizations.
for this purpose we need to set up a hypothetical probability model to

represent the data.
choose appropriate family of models determined by some parameters.
Then: fit the model to the data, estimate the parameters, check of goodness
of fit to the data, use the fitted model to enhance understanding of the
stochastic mechanism generating the series, use the model for prediction and
other applications of interest.
For the interpretation of the results (from both a statistical and an economic
point-of-view), it is important to recognize and eliminate the presence of
disturbing quantities like seasonal and/or other noisy components.
December 12, 2013
15/65
Basic Concepts
Some Definitions
Definitions
a time series (or stochastic process) in discrete time is a sequence of

real-valued random variables: {Xt : t Z}.
a time series model for the observed data {xt } is a specification of the joint
distributions of the time series {Xt : t Z} for which {xt } is postulated to be
a realization.
Remark
The definition above naturally extends to a multivariate vector of random variables.
December 12, 2013
16/65
Basic Concepts
Some Definitions
the laws of such a stochastic process are completely determined by the joint
distributions of every set of variables (Xt1 , Xt2 , . . . , Xtk ), k = 1, 2, . . .:
P [Xt1 xt1 , Xt2 xt2 , . . . , Xtk xtk ]
(3)
where < xt1 , xt2 , . . . , xtk < , k = 1, 2, . . .
a stochastic process is called a process of second order if

E[Xt2 ] < , t.
(4)
In this case, the laws are (at least partially) characterized only by the first two
moments (what are moment?)
December 12, 2013
17/65
Basic Concepts
Examples of Times Series

Some Zero-Mean Models:
iid noise: no trend or seasonal component, observations independent and

identically distributed with zero mean:
P [Xt1 xt1 , Xt2 xt2 , . . . , Xtk xtk ] = F (xt1 ) . . . F (xtk ),
where F () is the cumulative distribution function of X1 , X2 , . . .
A binary process: consider an iid random variables with

P [Xt = 1] = p, P [Xt = 1] = 1 p, with p =
1
.
2
Random Walk: (starting at zero) {St , t = 0, 1, 2, . . .} with

S0 = 0; St = X1 + X2 + . . . + Xt , for all t = 1, 2, . . . ,
where {Xt } is iid noise.
December 12, 2013
18/65
Basic Concepts
If, in addition, {Xt } is the binary process above, {St , t = 0, 1, 2, . . .} is called

a simple symmetric random walk.
Models with trend and seasonality:
Most of the time series examples presented above show a trend and/or a seasonal
component in the data:
Australian red wine sales
accidental deaths in the US
population in the US
in such cases a zero-mean model is clearly inappropriate. What to do?
December 12, 2013
19/65
Basic Concepts
Population in the US:

The earlier graph suggests trying the following model (no evident seasonal
component)
Xt = mt + Yt ,
where mt is a slowly changing function called the trend component and Yt has zero
mean.
What type of parametric representation do we choose for mt ?
mt = a0 + a1 t + a2 t2
(5)
where a0 , a1 , a2 are constants. Fitting by least squares: (in millions) we get

a
0 = 6.96, a
1 = 2.16 and a
2 = 0.651.
December 12, 2013
20/65
Basic Concepts
Accidental deaths in the US:

the graph suggests the presence of a strong seasonal pattern due to seasonally
varying factors (no evident trend component). This effect can be modeled by a
periodic component with fixed known period:
Xt = st + Yt ,
where st is a periodic function of t with period d and Yt has zero mean.
Convenient choice for st : sum of harmonics (or sine waves)
st = a0 +
k
X

aj cos(j t) + bj sin(j t) ,
j=1
where a0 , . . . , ak , b1 , . . . , bk are unknown parameters.
December 12, 2013
21/65
Basic Concepts
The parameters 1 , . . . , k are fixed frequencies, each being some integer multiple of
2/d.
(6)
We should thus choose k = 2 which will have periods twelve and six months.
Australian red wine sales:
From the earlier graph: both a trend and a seasonal pattern are visible.
build up a model with both trend and periodic components, of the form
Xt = mt + st + Yt ,
(7)
where mt , st and Yt are as defined before.
December 12, 2013
22/65
Basic Concepts

From the examples introduced above, we can derive a general strategy for time series
modeling.
1) Plot the series and examine the main features of the graph (trend and seasonal
patterns, outlying observations, . . .)
2) Remove the trend and seasonal components to get stationary residuals. When
needed, apply a preliminary transformation of the data.
3) Choose a model to fit the residual series, based on various sample statistics.
4) Use the fitted model to reach the final goals of the analysis.
Example: if the final goal is forecasting, use the model to forecast residuals,
then invert the transformations used in the first two steps to get forecasts of
the original series.
December 12, 2013
23/65
Basic Concepts
Introduction to Stationary Models
Introduction to Stationary Models

Idea: a time series {Xt , t Z} is said to be stationary if it has statistical properties
similar to those of the time-shifted series {Xt+h , t Z} for every integer h.
Definition: The stochastic process {Xt , t Z} is strictly stationary if the joint
distribution of every subset (Xt1 , . . . , Xtk ), k = 1, 2, . . . equals those of
(Xt1 +h , . . . , Xtk +h ), k = 1, 2, . . . for every integer h.
The problem with this definition is that it is not operational.
Definition: The stochastic process {Xt , t Z} is weakly (or second order)
stationary if
1) X (t) = E[Xt ] = is independent of t;
2) X (t + h, t) = Cov(Xt+h , Xt ) = E[(Xt+h X (t + h))(Xt X (t))] = (h)
is independent of t for each integer h.
December 12, 2013
24/65
Basic Concepts
Autocovariance and Autocorrelation functions
Autocovariance and Autocorrelation functions

Let us denote briefly for a stationary time series {Xt , t Z}
X (h) = X (h, 0) = X (t + h, t).
(8)
Note that in this case X (0) = Var(Xt ) independent of t and the process
{Xt , t Z} is homoskedastic.
Let us define now the autocovariance and autocorrelation functions. Note that
X (h) = X (h) (symmetry).
Definition: Let {Xt , t Z} be a stationary time series. The autocovariance
function (ACVF) of {Xt } at lag h is
X (h) = Cov(Xt+h , Xt ).
(9)
The autocorrelation function (ACF) of {Xt } at lag h is

Cov(Xt+h , Xt )
X (h)
=
.
X (h) = p
X (0)
Var(Xt+h )Var(Xt )
(10)
December 12, 2013
25/65
Basic Concepts
Examples
Examples
iid noise.
If {Xt } is iid noise and E[Xt2 ] = 2 < , then
2
,
X (t + h, t) =
0,
if h = 0,
if h =
6 0
(11)
which does not depend on t.

iid noise with finite second moment is stationary.
Notation: {Xt } IID(0, 2 ).
White noise.
{Xt } a sequence of uncorrelated random variables, with zero mean and variance 2 .
{Xt } is stationary with the same covariance function as the iid noise.
Notation: {Xt } WN(0, 2 ).
December 12, 2013
26/65
Basic Concepts
Examples
Note that every IID(0, 2 ) sequence is WN(0, 2 ) but not conversely.

The random walk.
If {St } is a random walk with {Xt } IID(0, 2 ), then E[St ] = 0 and
E[St2 ] = t 2 < for all t, and, for h 0, S (t + h, h) = t 2 .
Since S (t + h, h) depends on t, the series {St } is not stationary.
First-order moving average or MA(1) process.
Consider the series defined by the equation
Xt = + Zt + Zt1 , t Z,
(12)
where {Zt } WN(0, 2 ) and || < 1 a real-valued parameter. Then:
E[Xt ] = independent of t;
E[Xt2 ] = 2 + 2 (1 + 2 ) < ;
December 12, 2013
27/65
Basic Concepts
Examples
and
2
(1 + 2 ) ,
2 ,
X (t + h, t) =
0,
if h = 0,
if h {1; +1},
if | h |> 1
(13)
which does not depend on t.

{Xt } is stationary.
First-order autoregression or AR(1) process.
Let us assume that {Xt } is a stationary series satisfying the equations
Xt = c + Xt1 + Zt , t Z,
(14)
December 12, 2013
28/65
Basic Concepts
Examples
where {Zt } WN(0, 2 ), | |< 1, c and two real-valued parameters, and Zt is

uncorrelated with Xs for each s < t.
c
1
E[Xt ] =
E[Xt2 ] = 2 +
X (h) = X (h) = Cov(Xt , Xth ) = |h| X (0), and X (0) =
X (h) =
= constant;
X (h)
X (0)
2
12
< ;
2
;
12
= |h| , h Z.
In practice, we have to start by looking at observed data {xi }n

i=1 and then find a
link between the observed series and a good approximating model.
December 12, 2013
29/65
Basic Concepts
Sample Autocovariance and Autocorrelation functions
We can compute sample autocorrelation function (sample ACF), to assess the degree
of dependence in the data and then to select a model for the data that reflect this.
Definitions: Let {xi }n
i=1 be observations of a time series. The sample mean of
x1 , . . . , xn is computed as
n
1X
xt .
(15)
x=
n t=1
The sample autocovariance function at lag h is
(h) =
n|h|
1 X
(xt+|h| x)(xt x), n < h < n.
n t=1
(16)
The sample autocorrelation function at lag h is

(h) =
(h)
, n < h < n.
(0)
(17)
December 12, 2013
30/65
Basic Concepts
Sample Autocovariance and Autocorrelation functions Examples: iid Noise
Figure 8: 200 simulated values for an IID N (0, 1) noise.
Since (h) = 0 for h > 0 in the model, sample autocorrelations should be near 0.
Asymptotic theory: (h), h > 0 approximately IID N (0, 1/n) for n hlarge.
i
+1.96
Approximately 95% of sample autocorrelations should fall between 1.96

;
.
n
n
December 12, 2013
31/65
Basic Concepts
Sample Autocovariance and Autocorrelation functions Examples: a nonstationary example
Figure 9: Australian red wine sales data series
The sample autocorrelation function can be useed as an indicator of non-stationarity:
data with trend: |

(h)| exhibits slow decay as h increases;
data with seasonal component: |

(h)| exhibits similar behavior with same
periodicity.
December 12, 2013
32/65
Basic Concepts
Sample Autocovariance and Autocorrelation functions Examples: test of model residuals
Figure 10: ACF of Population in the US and ACF of residuals form a quadratic time trend
regression
We have seen that we can fit a model with a quadratic trend to this series.
how good is such a model (from a preliminary graphical inspection)?
look at the autocorrelation function of the residuals.
December 12, 2013
33/65
Basic Concepts
Estimation and Elimination of Both Trend and Seasonality

First step in the analysis of a time series: plot the data; when needed, transform the
data. Final purpose: get a stationary time series.
If there are any apparent discontinuities (sudden changes in level):
break the series into homogeneous segments.
If there are any outlying observations
check if there is any justification to discard them,
or try to model them
If any trend or seasonal components are evident:
represent the data as a Classical Decomposition Model.
December 12, 2013
34/65
Basic Concepts
In the classical decomposition, the realization of the process are modelled as:
Xt = mt + st + Yt ,
where mt is a slowly changing function, st is a function with known period d and Yt
is a cyclical component that is stationary, with E(Yt ) = 0.
Nonseasonal Model with Trend
Xt = mt + Yt , t = 1, . . . , n, where E[Yt ] = 0.
Method 1: Trend estimation
Final goal: find an estimate m
t for the trend component function mt .
December 12, 2013
35/65
Basic Concepts
a) Smoothing with a finite moving average filter:

Obtain {m
t } from {Xt } by application of a linear operator or linear filter
m
t =
aj Xtj , with some weights aj .
j=
For smoothing, consider the filter specified by the weights

aj = (2q + 1)1 , q j q,
q a nonnegative integer. This particular filter is a low pass filter: it takes the
data {Xt } and removes from it the rapidly fluctuating component to leave
the slowly varying estimated trend term {m
t }.
December 12, 2013
36/65
Basic Concepts
Then for q + 1 t n q,
Wt
q
X
1
Xtj
2q + 1 j=q
(18)
q
q
X
X
1
1
mtj +
Ytj
2q + 1 j=q
2q + 1 j=q
(19)
mt
(20)
and
m
t =
q
X
j=q
1
Xtj ,
2q + 1
assuming that mt is approximately linear over the interval [t q; t + q] and

that the average of the error terms over this interval is close to zero.
December 12, 2013
37/65
Basic Concepts
b) Exponential smoothing:
Compute the one-sided moving averages {m
t } as
m
t = Xt + (1 )m
t1 , t = 2, . . . , n, fixed [0, 1].
Take as initial value m
1 = X1 .
This model is often referred to as exponential smoothing, since for t 2
m
t =
t2
X
(1 )j Xtj + (1 )t1 X1 .
(21)
j=0
December 12, 2013
38/65
Basic Concepts
a) Smoothing by elimination of high-frequency components:

Using this method, the original series is smoothed by elimination of the
high-frequency components of its Fourier series expansion
( use spectral theory).
b) Polynomial fitting:
Assumption: the trend component is of a polynomial form (i.e. linear,
quadratic, cubic, . . .)
fit a polynomial function to the data {x1 , . . . , xn } to get estimates for
the coefficients (for example, by least squares).
December 12, 2013
39/65
Basic Concepts
Example. Strikes in the US
Number of strikes per year in the US. Time period: from 1951 to 1980.
December 12, 2013
40/65
Basic Concepts
Method 2: Trend Elimination by Differencing

Final goal: eliminate the trend term by differencing instead of smoothing like in
Method 1.
Let us define the lag-1 difference operator by
Xt = Xt Xt1 = (1 L)Xt ,
where L is the Lag (or Backshift) operator LXt = Xt1 . (Note: sometimes B is
used for Backshift and for difference operator)
Powers of the operator are defined recursively as
j (Xt ) = (j1 (Xt ), j 1, with 0 (Xt ) = Xt .
(22)
Polynomials in are manipulated in the same way as polynomial functions of real

variables (see the Section on the backward shift operator for more details).
December 12, 2013
41/65
Basic Concepts
Why difference data?

Any polynomial trend of degree k can be reduced to a constant by application of the
operator k .
possibility of getting a plausible realization of a stationary time series {k xt }
from the data {xt }.
In practice often the order k of differencing required is quite small: k = 1 or k = 2.
Differencing is not free! Comes at the costs of
of higher variances of the process
non-invertible MA models (we will see that later)
differencing needs to be applied with care
December 12, 2013
42/65
Basic Concepts
Example: Population in the US.
For population in the US already introduced above we find that differencing twice is
sufficient to produce a series with no apparent trend, ie.
{xt } {2 xt } = {xt 2xt1 + xt2 }.
December 12, 2013
43/65
Basic Concepts
Example: S&P500 Stock price index.
For daily US S&P500 Index values the trend evident in the data can be easily
eliminated by differencing once, that is:
{xt } {xt } = {xt xt1 }.
December 12, 2013
44/65
Basic Concepts
Classical Decomposition Model:
where E[Yt ] = 0, st+d
Xt = mt + st + Yt , t = 1, . . . , n,
Pd
= st and
j=1 sj = 0.
(23)
Method 1: Estimation of trend and seasonal components

Final goal: find estimates m
t , st for the trend and seasonal functions.
1) Estimate the trend by applying a moving average filter specially chosen to
eliminate the seasonal component and to dampen the noise:

(0.5xtq + P
xtq+1 + . . . + xt+q1 + 0.5xt+q )/d , if d = 2q even,
mt =
(2q + 1)1 qj=q xtj ,
if d = 2q + 1 odd.
December 12, 2013
45/65
Basic Concepts
2) Estimate the seasonal component:

for each k = 1, . . . , d compute the average wk of the deviations
{(xk+jd mk+jd ), q < k + jd n q};
P
Estimate sk = wk d1 di=1 wi , k = 1, . . . , d. Set: sk = skd , k > d.
3) Let
dt = xt st , t = 1, . . . , n,
(24)
be the deseasonalized data.

Re-estimate the trend from the deseasonalized data {dt } m
t.
December 12, 2013
46/65
Basic Concepts
The estimated cyclical component yt is then given by

yt = xt m
t st , t = 1, . . . , n.
(25)
Remark: Note that the re-estimation of the trend in step 3. above is done in order to
have a parametric form for the trend that can be used for simulation and prediction.
Method 2: Elimination of trend and seasonal components by differencing
Define the lag-d differencing operator d by
d Xt = Xt Xtd = (1 Ld )Xt .
(26)
December 12, 2013
47/65
Basic Concepts
Apply the operator d to the classical decomposition model

d Xt = mt mtd + Yt Ytd ,
(27)
where st has period d.
st thus drops out because st = std .
The obtained model in (27) has a trend and a noise component.
Eliminate the trend component by differencing as before.
December 12, 2013
48/65
Basic Concepts
Example: Accidental deaths in the US.
December 12, 2013
49/65
Basic Concepts
Testing the Estimated Residual Sequence

The objective of the data transformations described above is to produce a sequence
of stationary residuals.
Next step: find a model for the residuals.
no dependence in the residual series: residuals come from an iid process, no

further modeling needed;
significant dependence among residuals: look for a more complex stationary

model.
To determine this: use simple tests for checking the hypothesis that the residuals are
observed values of iid random variables.
December 12, 2013
50/65
Basic Concepts
a) Sample autocorrelation function.

For large n, the sample autocorrelation function of an iid sequence
Y1 , . . . , Yn with finite variance is N (0, 1/n).
if y1 , . . . , yn is a realization of such an iid sequence, about 95% of the
; + 1.96
].
sample autocorrelations should fall between the bounds [ 1.96
n
n
b) Portmanteau type test.
Let us consider other statistics for the sample autocorrelations (j):
QP = n
h
X
2 (j) (Portmanteau test)
(28)
j=1
QLB = n(n + 2)
h
X
2 (j)
(Ljung-Box test)
nj
j=1
(29)
December 12, 2013
51/65
Basic Concepts
QML = n(n + 2)
h
X
2W W (j)
(McLeod-Li test)
nj
j=1
(30)
where W W (j) are the sample autocorrelations of the squared data.

Under the assumption that the residuals are a finite-variance iid sequence,
the three statistics are 2 (h) (Chi-squared distributed).
The hypothesis of iid data is then rejected at level if Q() > 21 (h).
c) The turning point test.
Let y1 , . . . , yn be a sequence of observations. We say that there is a turning
point at time i, 1 < i < n if
December 12, 2013
52/65
Basic Concepts
(i) yi1 < yi and yi > yi+1 or

(ii) yi1 > yi and yi < yi+1 .
If NT P is the number of turning points of an iid sequence of length n, then
NT P = E[NT P ] = 2(n 2)/3;
2
N
= V (NT P ) = (16n 29)/90.
TP
A large value of NT P NT P indicates that the series is fluctuating more

rapidly than expected for an iid sequence.
On the other side: a value of NT P NT P much smaller than zero
indicates a positive correlation between neighboring observations.
December 12, 2013
53/65
Basic Concepts
2
In fact: for an iid sequence with large n: NT P N (NT P , N
).
TP
Test: reject the iid hypothesis at level if

| N T P N T P |
> 1/2 .
NT P
(31)
d) The difference-sign test.

Let NS be the number of values of i such that yi > yi1 , i = 2, . . . , n. For
an iid sequence we have:
NS = E[NS ] = (n 1)/2;
2
N
= V (NS ) = (n + 1)/12,
S
2
and for large n: NS N (NS , N
).
S
December 12, 2013
54/65
Basic Concepts
A large positive (or negative) value of NS NS indicates the presence of a

trend in the data.
Test: reject the hypothesis of no trend in the data if
| N S N S |
> 1/2 .
NS
(32)
This test must be taken with caution: observations exhibiting a strong

cyclical component will pass the difference-sign test!
e) Fitting an autoregressive moving average type models.
to be discussed later.
December 12, 2013
55/65
Basic Concepts
f) Checking for normality.

Draw a Gaussian qq-plot to verify whether the data may be assumed to
come from a Gaussian iid sequence.
g) The rank test.
This test is particular useful to detecting a linear trend in the data.
Define NP to be the number of pairs (i, j) such that
yj > yi and j > i, i = 1, . . . , n 1.
Note that there is a total of
1
n(n
2
(33)
1) pairs such that j > i.
If {Y1 , . . . , Yn } is an iid sequence:

NP = E[NP ] = n(n 1)/4
(34)
December 12, 2013
56/65
Basic Concepts
2
N
= V (NP )
P
= n(n 1)(2n + 5)/72,

2
and for large n: NP N (NP , N
).
P
A large positive (negative) value of NP NP indicates the presence of an

increasing (decreasing) trend in the data.
Test: reject the hypothesis that {yt } is a sample from an iid sequence if
| N P N P |
> 1/2 .
NP
(35)
December 12, 2013
57/65
Basic Concepts
Example: Accidental deaths in the US.cont.
December 12, 2013
58/65
Basic Concepts
Example: Accidental deaths in the US. cont.
December 12, 2013
59/65
Basic Concepts
The Lag (or Backshift) Operator

Define as before the Lag (or Backshift operator) as: LXt = Xt1 .
Properties:
1) linearity: L(Xt + Yt ) = Xt1 + Yt1 and L(Xt ) = Xt1 , a constant;
2) powers: Lk Xt = Xtk , k = 1, 2, . . .;
3) inverse: L1 Xt1 = Xt .
Xt = Xt Xt1 = (1 L)Xt
(36)
Xt = Xt1 + Yt (1 L)Xt = Yt
(37)
Lc = c, where c is a constant.
(38)
December 12, 2013
60/65
Basic Concepts
Polynomials in the Lag (or Backshift) Operator

Let us consider the expression
a0 Xt + a1 Xt1 + a2 Xt2 + . . . + an Xtn ,
where ai are constant coefficients.
We can rewrite it using the properties of the backward shift operator L as
(a0 + a1 L + a2 L2 + . . . + an Ln )Xt = a(L)Xt
a polynomial of degree n (that can also be equal to ) in L.
All the classical operations used for polynomials can be applied to a(L)
December 12, 2013
61/65
Basic Concepts
evaluation in L = 1:
a(1) = a0 + a1 + . . . + an =
n
X
ai
(39)
i=0
first derivative:
d
a(L) = a0 (L) = a1 + 2a2 L + 3a3 L2 + . . . + nan Ln1 .
dL
Then
a0 (1) = a1 + 2a2 + . . . + nan =
n
X
iai .
(40)
(41)
i=1
A polynomial a(L) is invertible if all the solutions of the characteristic equation

a0 + a1 z + . . . + an z n = 0
(42)
lie outside the unit circle.
December 12, 2013
62/65
Exercises
1) Let {Zt } be a sequence of independent normal random variables, each with mean 0 and variance
2 , and let a, b and c be constants. Which, if any, of the following processes are stationary? For
each stationary process specify the mean and autocovariance function.
a) Xt = a + bZt + cZt2 ;
b) Xt = Zt cos(ct) + Zt1 sin(ct);
c) Xt = a + bZ0 ;
d) Xt = Zt Zt1 .
2) Let {Xt } be a moving average process of order 2 given by
Xt = Zt + Zt2 , where Zt WN(0, 1).
(43)
a) Find the autocovariance and autocorrelation functions for this process when = 0.8.
P
b) Compute the variance of the sample mean X 4 = 14 4j=1 Xj .
c) Repeat b) when = 0.8 and compare your answer with the result obtained in b)
December 12, 2013
63/65
Exercises
3) Let {Xt } be the AR(1) process defined as:

Xt = Xt1 + Zt , WN .
(44)
a) Compute the variance of the sample mean X 4 when = 0.9 and 2 = 1.

b) Repeat a) when = 0.9 and compare your answer with the result in a).
4) Consider the simple moving average filter with weights
aj = (2q + 1)
a) If mt = c0 + c1 t, show that
Pq
j=q
, q j q.
(45)
aj mtj = mt .
b) If Zt , t Z, are independent random variables with mean 0 and variance 2 , show

that the moving average
q
X
At =
aj Ztj
j=q
is small for large q in the sense that E[At ] = 0 and V (At ) =
2
.
2q+1
December 12, 2013
64/65
Exercises
5) Let {Yt } be a stationary process with mean zero and let a and b be constants. If
Xt = a + bt + st + Yt ,
(46)
where st is a seasonal component with period 12, show that

12 Xt = (1 L)(1 L
12
)Xt
(47)
is stationary and express its autocovariance function in terms of that of {Yt }.

6) Let us consider an invertible polynomial a(L) with finite a(1). Show that such a polynomial can be
rewritten as
a(L) = a(1) + (1 L)g(L),
(48)
where
g0 = a0 a(1)
g1 = a1 + g0
(49)
g2 = a2 + g1
etc.
Compute g(1).
December 12, 2013
65/65

Basic Time Series

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Basic Time Series

Uploaded by

Copyright:

Available Formats

Linear Time Series Analysis

Lecture 1: Some Basic Time Series Concepts

December 12, 2013

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

continuous-time time series, when observations are recorded continuously over

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

rolling means: instead of yt plot

where d is an odd positive integer.

exponential smoothing: instead of yt plot

with y1 = y1 and 0 < a < 1.

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

generally, analysis is performed on stationary time series. If the time series is

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

if it exists, take a linear combination of two (or more) stochastic processes

the series are then said to be cointegrated

least-squres is valied for fitting but adjustments to standard errors need to be

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

not apparent trend

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

need to determine the unknown signal component smooth the data by

eliminate high-frequency components spectral analysis.

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

HELP Index: random fluctuation around a slowly changing level.

IP: evident upward trend, yearly seasonality component.

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

upward trend in the Index values is evident,

log-returns show that one can eliminate trend in Index by differencing.

heteroskedasticity in the return series need model for volatility

Daniel Buncic (University of St. Gallen)

Lecture 1: Linear Time Series Analysis

December 12, 2013

Examples of Time Series

as above for the index.