PROBABILITY, RANDOM VARIABLES, AND STOCHASTIC PROCESSES

FOURTH EDITION

Athanasios Papoulis
University Professor Polytechnic University

S. Unnikrishna Pillai
Professor of Electrical and Computer Engineering Polytechnic University

Me Graw Hill
Boston Burr Ridge, IL Dubuque, IA Madison, Wl New York San Francisco St. Louis Bangkok Bogota Caracas Kuala Lumpur Lisbon London Madrid Mexico City Milan Montreal New Delhi Santiago Seoul Singapore Sydney Taipei Toronto

CONTENTS

Preface PART I PROBABILITY AND RANDOM VARIABLES Chapter 1 The Meaning of Probability
1-1 Introduction / 1-2 The Definitions / 1-3 Probability and Induction / 1-4 Causality Versus Randomness

ix 1 3

Chapter 2

The Axioms of Probability

2-1 Set Theory / 2-2 Probability Space / 2-3 Conditional Probability / Problems

15

Chapter 3

Repeated Trials
3-1 Combined Experiments / 3-2 Bernoulli Trials / 3-3 Bernoulli's Theorem and Games of Chance / Problems

46

Chapter 4

The Concept of a Random Variable

4-1 Introduction / 4-2 Distribution and Density Functions / 4-3 Specific Random Variables / 4-4 Conditional Distributions / 4-5 Asymptotic Approximations for Binomial Random Variable / Problems

72

Chapter 5

Functions of One Random Variable

5-1 The Random Variable g(x) I 5-2 The Distribution ofg(x) / 5-3 Mean and Variance / 5-4 Moments / 5-5 Characteristic Functions / Problems

123

Chapter 6

Two Random Variables

6-1 Bivariate Distributions / 6-2 One Function of Two Random Variables / 6-3 Two Functions of Two Random Variables / 6-4 Joint Moments / 6-5 Joint Characteristic Functions / 6-6 Conditional Distributions / 6-7 Conditional Expected Values / Problems

169

VI

CONTENTS

Chapter 7

Sequences of Random Variables

7-1 General Concepts / 7-2 Conditional Densities, Characteristic Functions, and Normality / 7-3 Mean Square Estimation / 7-4 Stochastic Convergence and Limit Theorems / 7-5 Random Numbers: Meaning and Generation / Problems

243

Chapter 8

Statistics
8-1 Introduction / 8-2 Estimation / 8-3 Parameter Estimation / 8-4 Hypothesis Testing / Problems

303

PART II Chapter 9

STOCHASTIC PROCESSES General Concepts

9-1 Definitions / 9-2 Systems with Stochastic Inputs / 9-3 The Power Spectrum / 9-4 Discrete-Time Processes / Appendix 9A Continuity, Differentiation, Integration / Appendix 9B Shift Operators and Stationary Processes / Problems

371 373

Chapter 10 Random Walks and Other Applications

10-1 Random Walks / 10-2 Poisson Points and Shot Noise / 10-3 Modulation / 10-4 Cyclostationary Processes / 10-5 Bandlimited Processes and Sampling Theory / 10-6 Deterministic Signals in Noise / 10-7 Bispectra and System Identification / Appendix 10A The Poisson Sum Formula / Appendix 10B The Schwarz Inequality / Problems

435

Chapter 11 Spectral Representation

11-1 Factorization and Innovations / 11-2 Finite-Order Systems and State Variables / 11-3 Fourier Series and Karhunen-Loeve Expansions / 11-4 Spectral Representation of Random Processes / Problems

499

Chapter 12 Spectrum Estimation

523

12-1 Ergodicity / 12-2 Spectrum Estimation / 12-3 Extrapolation and System Identification / 12-4 The General Class of Extrapolating Spectra and Youla's Parametrization / Appendix 12A Minimum-Phase Functions / Appendix 12B All-Pass Functions / Problems

Chapter 13 Mean Square Estimation

13-1 Introduction / 13-2 Prediction / 13-3 Filtering and Prediction / 13-4 Kalman Filters / Problems

580

Chapter 14 Entropy

'

629

14-1 Introduction / 14-2 Basic Concepts / 14-3 Random Variables and Stochastic Processes / 14-4 The Maximum Entropy Method / 14-5 Coding / 14-6 Channel Capacity / Problems

CONTENTS

Vll

Chapter 15

Markov Chains
15-1 Introduction / 15-2 Higher Transition Probabilities and the Chapman-Kolmogorov Equation / 15-3 Classification of States / 15-4 Stationary Distributions and Limiting Probabilities / 15-5 Transient States and Absorption Probabilities / 15-6 Branching Processes / Appendix 15A Mixed Type Population of Constant Size / Appendix 15B Structure of Periodic Chains / Problems

695

Chapter 16

Markov Processes and Queueing Theory

16-1 Introduction / 16-2 Markov Processes / 16-3 Queueing Theory / 16-4 Networks of Queues / Problems

773

Bibliography Index

835 837