Introduction to Stochastic Processes

U Dinesh Kumar

Lecture Outline

Introduction and definition of SP. Characteristics of SP. Types of SP.

Strength and Weakness of SP models.

In mathematics, you don't understand things. You just get used to them - J von Neumann

Stochastic Processes - Definition

A stochastic process is a model that evolves in time or space subject to probabilistic laws.
A stochastic process {X(t), t T} is a collection of random variables indexed by t. The index t is usually assumed to be time and the random variable X(t) is the state of system at time t.
If the index set T is countable set, then the process X(t) is a discrete-time stochastic process. If T is continuous, then X(t) is called continuous time process.

Importance of Stochastic Process

Theory of stochastic processes aims to model the interaction of chance and time. It is all most impossible to find a problem in business which does not change with time and has no uncertainty. Often, SP is the only tool which can model realistic problems in business and management.

Path of SP X(t)

The values taken by SP X(t) for various values of t is called the path of the SP X(t)
a random variable for each fixed t

X(t)

a sample path

time

State Space

The range of possible values of SP, X{t}, is called the state space. The state space can be either discrete or continuous. Discrete State Space Number of customers Credit Rating Continuous State Space Market Share Stock Price

Examples of SP

Cash flow Market Share Stock Price Customer Retention Customer Satisfaction Inventory Level Commodity Price Product and service demand Availability of engineering systems

Stochastic Processes: Independent Increment

A continuous time SP {X(t)} is said to have independent increments if for all t0 < t1 < t2 < < tn: the random variables:

X(t1) X(t0), X(t2) X(t1), X(t3) X(t2), , X(tn) X(tn1) are independent. In an independent increment process, the change in the non-overlapping time intervals are independent.

Stationary Processes

A SP is said to have stationary increments if X(t+s) X(t) has same distributions for all t. In a stationary process, the distribution of change between two points depends only on the distance between the two points.

Types of Stochastic Processes

Poisson Process Renewal Process Markov Process Martingales Random Walks Brownian Motion Process

Strength of SP

The element of randomness exists in any process and thus you need a tool that can model changing phenomena.

All most all problems in finance, marketing and operations have random element and the distribution of randomness changes over a period of time.

Weakness of SP

The SP models in many cases can be intractable. Many SP models are still unsolved.

Recommended Readings
Ross, S M, Introduction to Probability Models, Academic Press, 2006.
E Cinlar, Introduction to Stochastic Processes, Prentice Hall, 1975

H C Tijms, Stochastic Models An Algorithmic Approach, John Wiley, 1994.