Digital and Kalman Filtering: An Introduction to Discrete-Time Filtering and Optimum Linear Estimation, Second Edition

Index

Preface to the first edition

The availability of digital computers has stimulated the use of digital signal processing (or time series analysis) in many diverse fields covering engineering and also, for example, medicine and economics. Therefore, the terms digital filtering, Kalman filtering, and other types of processing, appear quite often in the present-day professional literature. The aim of this two-part book is to give a relatively simple introduction to digital and Kalman filtering. The first part covers a filtering operation as normally understood in electrical engineering and specified in the frequency-domain. The second part deals with the filtering of noisy data in order to extract the signal from noise in an optimum (minimum mean-square error) sense.

Digital filtering, as used in the title, refers to part 1 of the book, but the subtitle specifies it more closely as an introduction to discrete-time filtering which is a common theory for digital or other types of filter realizations. The actual realization of discrete-time filters, not discussed here, can be done either in sampled-data form (using bucket-brigade or charge coupled devices), or in digital form (using binary logic circuitry). Alternatively, discrete-time filters, described in terms of difference equation algorithms, can be handled on digital computers.

Kalman filtering, as used in the title, refers to part 2 of the book, and again the subtitle describes it more closely as an introduction to linear estimation theory developed in discrete-time domain. Although this part deals initially with some digital filter structures, it develops its own terminology. It introduces the criterion of the minimum mean-square error, scalar and vector Wiener and Kalman filtering. However, the main and most practical topic is the Kalman filtering algorithm which in most applications requires the use of digital computers.

Most of the material has been used by the author in postgraduate courses over the past five years. The presentation is in ‘tutorial’ form, but readers are assumed to be familiar with basic circuit theory, statistical averages, and elementary matrices. Various central topics are developed gradually with a number of examples and problems with solutions. Therefore, the book is suitable both for introductory postgraduate and undergraduate courses.

The author wishes to acknowledge helpful discussions with Dr J. A. Edwards of this School, who also helped with some of the computer programs.

SMB

1979

Preface to the second edition

Over a period of time a number of communications have been received about the first edition of this book. Some were comments about contents and others were corrections regarding the examples and problems. These have been taken into account in the second edition, and the author wishes to thank all the readers for their contributions.

In the second edition some material has been reorganized and some new topics have been added, In chapter two, the main new addition is a section on the graphic method for frequency response computations. In chapter three, some sections have been rearranged, and also two new sections added. One deals with the FIR filter design by the frequency sampling method and another one introduces the equiripple FIR filter design. In chapter four, more information is given about the frequency transformations, quantization effects, and also the wave digital filters have been introduced. In chapter five, two new sections have been added: circular convolution, and an introduction to multirate digital filters.

In part 2, chapters six and seven have been considerably rearranged. New material has been added in chapter six: IIR Wiener filters, and adaptive FIR filters. Also, in both chapters Wiener and Kalman filters have been interpreted as lowpass filters with automatically controlled cut-off, hence behaving as intelligent filters. In chapter eight, a new section has been added dealing with practical aspects of Kalman filtering. Similarly, chapter nine has been changed by replacing the vector Wiener filter example with two new examples showing practical aspects in setting up the stage for Kalman filter application.

The author wishes to thank referees for their valuable suggestions. It would be nice, in future, to link the examples and problems to the Matlab software package.

SMB

1994

Part 1 – Digital filtering

Introduction

Digital filtering is used here as a well-established title, but with the reservation that we are dealing only with time sequences of sampled-data signals. However, the fundamental theory presented for discrete-time signals is general and can also be applied to digital filtering.

It is important to clarify the terminology used here and in the general field of signal processing. The analogue or continuous-time signal means a signal continuous in both time and amplitude. However, the term continuous-time implies only that the independent variable takes on a continuous range of values, but the amplitude is not necessarily restricted to a finite set of values, as discussed by Rabiner (1). Discrete-time implies that signals are defined only for discrete values of time, i.e. time is quantized. Such discrete-time signals are often referred to as sampled-data or analogue sample signals. The widely-used term digital implies that both time and amplitude are quantized. A digital system is therefore one in which a signal is represented as a sequence of numbers which take on only a finite set of values.

It is also important to clarify the notation used here. In mathematics the time increments or decrements are denoted by Δt, but in digital filtering the sampling time interval T is generally used. A sample of input signal at time t = kT is denoted as x(k), where T is neglected (or taken as unity) and k is an integer number; similarly, for the output we have y(k). In many papers and textbooks, particularly mathematical ones (difference equations), the notation is xk, yk. We use here the notation x(k), y(k) for the following reasons:

(i) it is a direct extension of the familiar x(t), y(t) notation used for the continuous-time functions;

(ii) it is suitable for extension to the state-variable notation, where the subscripts in x 1 (k), x 2 (k) . . . refer to states;

(iii) it is more convenient for handling complicated indices, for example x ( N – ).

It will be seen later that digital filtering consists of taking (usually) equidistant discrete-time samples of a continuous-time function, or values of some discrete-time process, and performing operations such as discrete-time delay, multiplication by a constant and addition to obtain the desired result.

The first chapter introduces discrete-time concepts using some simple and familar analogue filters, and also shows how a discrete-time description can arise directly from the type of operation of a system, e. g. radar tracking. The z-transform is then introduced as a compact representation of discrete-time sequences, and also as a link with the Laplace transformation. The second chapter expands the basic concepts established in the first chapter, dealing first with the time response of a discrete-time system (difference equations). Then we introduce and discuss the transfer function, inversion from z-variable back to time-variable, and the frequency response of a digital filter. This chapter ends with the realization schemes and classification of digital filters into the basic nonrecursive and recursive types, whose design techniques are presented in the third and fourth chapters respectively. In both cases design examples and computer calculated responses are given to illustrate various design methods. In the fifth chapter we return to some of the basic relationships introduced in the first chapter, and deal with two important topics in discrete-time processing. First we develop the discrete Fourier series representation of periodic sequences, which enables formulation of the discrete Fourier transform (DFT) for aperiodic finite sequences. The second topic is the inverse filter, which is an important concept used in many fields for removal or reduction of undesirable parts of a sequence. The concept of minimum error energy is introduced as an optimization technique for the finite length inverse filter coefficients. At the end of each chapter a number of problems is given with solutions at the end of the book.

It is of interest to mention that the approach used in this book is a form of transition from the continuous- (or analogue) to discrete-time (or digital) systems, since most students have been taught electrical engineering in terms of continuous-time concepts. An alternative approach is to study electrical engineering directly in terms of discrete-time concepts without a reference to continuous-time systems. It appears that the discrete case is a natural one to the uninhibited mind, but special and somewhat mysterious after a thorough grounding in continuous concepts, as discussed by Steiglitz (2), p. viii.

1

Introduction to discrete-time filtering

1.0 Introduction

In continuous-time, the filtering operation is associated with RC or LC type of circuits. Therefore, in the first section of this chapter, we consider two simple filtering circuits (RC and RLC) described in continuous-time by differential equations, and we find their discrete-time equivalents, i.e. difference equations. There are also situations in which difference equations are obtained directly, as illustrated in section 1.2. In continuous-time we usually represent differential equations in the complex frequency s-domain by means of Laplace transformation, and from these we obtain the frequency response along the s = jω axis. Similarly, difference equations in discrete-time are transformed into z-domain using z-transformation which is briefly introduced in the third section of this chapter. The relationship between z and s is then established in section 1.4.

1.1 Continuous and discrete-time analysis

We are all familiar with the description of continuous-time dynamic systems in terms of differential equations. As an introduction to the discrete-time description and the process of filtering we consider a few differential equations and transform them into their discrete-time equivalents, i.e. difference equations.

Fig. 1.1 (a) Simple first-order RC filter; (b) solution for unit step input

Consider the typical first-order RC filter in fig. 1.1(a) where x and y represent the input and output voltages respectively. For this simple network x and y are related by the differential equation

The solution, for a unit step input and zero initial condition, is shown in fig. 1.1(b). To derive the discrete-time equivalent for equation 1.1, we use the method of backward differences, described by Hovanessian et al. (3), and obtain

we have

Using the approximation

where we have neglected terms of higher order, we obtain

with a0 = Δt/RC and b1 = 1 – a0. Note that we have changed to sample notation, as discussed in the introduction, with k representing the discrete integer time parameter instead of t = kΔt. The above result enables us to draw fig. 1.2(a) which is the discrete-time equivalent of the continuous-time system shown in fig. 1.1(a). In fig. 1.2(a), triangles are used to represent multiplication by the factor written beside them, rectangles to represent delay units denoted also by D(=Δt), and circles to represent addition. Choosing numerical values a0 = Δt/RC = 0.1, b1 = 1 – a0 = 0.9, and y (-1) = 0 as the initial condition, we obtain fig. 1.2(b), where the first ten points have been calculated using a slide rule.

Fig. 1.2 (a) Discrete-time representation of fig. 1.1 (a); (b) solution for unit step input

A typical second-order differential equation is

where again x and y refer to the input and output respectively. Such an equation describes, for example, the LRC circuit in fig. 1.3(a) with σ = R/2L The solution for the unit step input is given in fig. 1.3(b) for the underdamped case when

The discrete-time form of equation 1.3 can be shown to be

Fig. 1.3 (a) LRC filter; (b) solution for unit step input

where backward differences have been used, and the coefficients are functions of σ and ω0 (see section 1 of the appendix). The discrete-time operational scheme for the second-order difference equation 1.4 is shown in fig. 1.4, with the same notation as in fig.1.2(a). The unit step response in discrete-time has not been calculated, but the interested reader may do it as an exercise.

Fig. 1.4 Discrete-time representation of fig. 1.3(a)

The input x(t) and output y(t) variables