Toeplitz matrices and more

First studied by Toeplitz in 1901. tij = tij . Defined

r0
r1
..

.
r0
p1
T = .
... ...
..
pn1 p1

by 2n 1 parameters.

rn1
..
.

r1
r0

Applications: computation of splines, time series analysis, Markov chains, queuing theory,
signal and image processing, PDE solutions.

Properties
1. A Toeplitz matrix T is persymmetric, so is its inverse.
Proof. Let En = f liplr(In ) (exchange matrix). Then T = En T T En (persymmetric).
Then T 1 = En T T En .
2. All Toeplitz matrices commute asymptotically, or diagonalize in the same basis when
n .
3. Their eigenvectors are sines and cosines.
4. T is closely related to FFT. Let a Fourier operator (DFT matrix)
F = f f t(eye(n))/sqrt(n)
F CF 1 = T D0 , D0 = diag(1, , 2 , . . . , n1 ), = ei/n
5. T can be converted into a circulant matrix with the size roughly doubled and 1st col
v = (pT rT )T . A circulant matrix C can be diagonalized by F :
F CF 1 = D = diag(F v).
6. Circular convolution circulant Toeplitz matrix-vector product.
7. Toeplitz matrix vector in O(n log n).

x
T B
x
Tx
C
=
=
0
B T
0
Bx
8. Displacement structure
T
T
T = u:1:2 v:,1:2
Z1 T Z1

Z =

.
.

.
1

Z can be diagonalized by FFT.

T
D1 C CD1 = u:,1:2 v:,1:2

D1 = diag(1, 2 , 4 , . . . , 2(n1) )
D1 = D1

!
T
ui,1:2 vj,1:2
C=
di fj

Solution
Method
Fast and stable
Fast and unstable
Superfast preconditioner
Superfast and stable
Superfast and unstable
New superfast and stable

Cost
20n2
3n2
O(n log n)
O(n2 + n log2 )
O(n log2 n)
O(n log n)

Storage
12 n2
4n
O(n)
O(n log2 )
O(n)
O(n)

Fast: Levinson-Durbin, Trench ...

Fast stable: Nagy, Chandrasekaran, Sayed, Gohberg, Kailath, Olshevsky, Gu ...
Superfast Preconditioners: Benzi, R. Chan, Nagy, Plemmons, T. Chan, Strang, Yeung,
Di Benedetto, Jin, Kailath, Olshevsky, Ku, Kuo, Strela, Tyrtyshnikov, ...
Superfast: Martinsson, Tygert, Rokhlin, Ammar, Gragg, Stewart, Codevico, van
Barel, Heinig, Chandrasekaran, Gu, Xia, Zhu ...

2.1

Levinson (1947)

Consider SPD case first. Scale diags to be 1. Write

Tk E k r
v
b1:k
Tk+1 y =
=
r T Ek 1

bk+1
Suppose Tk x = b1:k and y = Tk1 r are solved. Yule-Walker eq.:
Tk y = r
Due to the persymmetry, Tk1 Ek = Ek Tk1 . Then

v
Tk Ek r
= b1:k =

v = Tk1 (b1:k Ek r) = x Tk1 Ek r = x Ek (Tk1 r) = x + Ek y

T T
v
r Ek 1
= bk+1 =

= bk+1 rT EkT v = bk+1 rT EkT (x + Ek y) = bk+1 rT EkT x /(1 + rT y)

2

Here, 1 + rT y > 0 since Tk+1 is SPD (Golub p.194).

Cost: 4n2 and can be improved.

2.2

Inversion (Trench)

Tn1

A Er
rT E 1

B v
vT

Then

A Er
T
r E 1

0
1

Av = Er, rT Ev + = 1
Use the Yule-Walker system Ay = r.
= 1 rT Ev = 1 + (Ay)T Ev = 1 + y T EAv = 1 y T EEr
= 1 y T r
1
1 + rT y
1
v = A Er = EA1 r = rEy
= =

These give the last column/row of Tn1 .

Then
AB = Erv T = In1
B = A1 +

vv T

vi vj
bij = A1 ij +

1
vi vj
= A nj,ni +
(symmetry about antidiag)

vnj vni vi vj
= bnj,ni
+
.

Thus, this gives the symmetric border.

Repeat from boundary to center: row/cols n, 1, n 1, 2, . . . Cost: 13n2 /4.

2.3

Nonsymmetric case

For T defined by vectors r, p, Use

TkT y = r, Tk w = p, Tk x = b.

2.4

Stability

Theorem 1 (Cybenko) The Levinson and Trench algorithms are weakly stable for SPD
Toeplitz matrices.
Let k be in Levinson for Yule-Walker, and
T
(rk+1 + r1:k
Ek y1:k )
.
ek

T
ek = 1 + r1:k
y1:k , k =

Then the condition number

||T

||1

n1
Y
1

1 + |k |
1 |k |

Large for k close to 1. A rounding error committed at the jth stage becomes magnified by
a factor of ej1 ej1en1 at the nth stage.

Factorization/Schur algorithms

Each time, make one lower/upper diagonal zero, from the first super-/sub-diagonal to the
nth.

Bareisss factorization algorithm [1]

4

QR algorithm

T =

R=

T0 u
=
= QR
vT t0

zT
Rt z
=
Rb
0 rnn

t 0 uT
v T0
r11
0

Set
T T T = RT R
RbT Rb = RtT Rt + uuT vT v z T z
Write
R1T R1 = RtT Rt + uuT updating
R2T R2 = R1T R1 vT v updating
RbT Rb = R2T R2 z T z updating
Where R1 , R2 are upper triangular. For updating problems, there exists a product Q of
Givens rotations s.t.

Rt
R1
Q
=
uT
0
For downdating problems, there exists a product Y of Hyperbolic ratations s.t.

R1
R2
Y
=
vT
0

References
[1] E. H. Bareiss, Numerical solution of linear equations with Toeplitz and vector Toeplitz
matrices, Numer. Math. 13 (1969), pp. 404424.
[2] G. Cybenko, The numerical stability of the Levinson-Durbin algorithm for Toeplitz
systems of equations, SIAM I. Sci. Statist. Comput. 1 (1980), pp. 303319.
[3] J. R. Bunch, The weak and strong stability of algorithms in numerical linear algebra,
Linear Algebra Appl., 88-89 (1987), pp. 4966.
[4] G. Golub and C. F. Van Loan, Matrix Computations, Johns Hopkins.
[5] N. Levinson, The Wiener RMS error criterion in filter design and prediction, J. Math.
Phys., 25 (1947), pp. 261278.
[6] J. Nagy, Fast Inverse QR Factorization for Toeplitz Matrices, SIAM J. Sci. Comput., 14
(1993), pp. 11741193.