GSN3113: Probability and

Stochastic Process
Lecture 2
SET Theory & Classical
Probability
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Let look back at the Previous Lecture

REVIEW OF PREVIOUS
LECTURE
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Randomness
Randomness or Random Phenomena arise
because of
Our partial ignorance of the generating mechanism
The laws governing the phenomena may be
fundamentally random
Our unwillingness to carry our exact analysis
because its not worth the trouble.

What other factor/reasons

randomness in a process?
GSN3113: Probability and Stochastic Process UCP Fall 2015

may

cause
Dr. Asim Baig

Back to Probability Theory

Probabilities are useful in situations that involve
randomness.
A probability is a number used to describe how likely
something/some event to occur.
Probability is the study of probabilities
It is the art of being certain how uncertain you are

If an event is certain to happen, it is given a probability of

1.
If it is certain not to happen, it has probability of 0.
Probabilities can be expressed as factions, as decimal
numbers or as percentages.
If you toss a coin, the probability of getting heads is , which is same
as 0.5, which is same as 50%
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Long-run Frequency
Interpretation
If the probability of an event A in some actual physical experiment is
p, then we believe that if the experiment is repeated independently
over and over again then the fraction of times the event occurs will be
close to P(A).
This is called the law of large numbers (LLN)
Let A be one of the events of a random experiment. If we conduct a
sequence of n independent trials of this experiment, and if the event A
occurs N(A,n) out these n trials then

is called the relative frequency of event A in these n trials

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Long run frequency

Interpretation
Long run frequency interpretation can be stated in
terms of relative frequency as

Which means that given enough trials, the relative

frequency will eventually become close to P(A)
i.e. you can calculate probability of event A
occurring by performing large number of trials.
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Coin Tossing: Relative

Frequency
If a fair coin is flipped a large number of
times, the proportion of heads will tend to get
closer to as the number of tosses increases.

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Collecting Stuff.

REVIEW OF SET THEORY

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Set Theory Introduction

The mathematical basis of probability is the
Set Theory.
We will start with a review of Set Theory so as
to refresh the previous knowledge.
There is a high Probability you do not
remember most of the set theory.
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Introduction to Sets
Set is a collection of things. We use capital letters to denote
sets.
The things that together make up the set are elements. We use
small letter to denote set elements.
For Example:

We can have a set A with elements x, y and z.

The symbol denotes set inclusion
Thus means x is an element of set A.
Similarly is the opposite of
Thus means c is not an element of A

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Generation of Sets.
There are many ways to define a set.
First is to simply list all the elements

In Engineering we frequently use mathematical rules to define sets.

Or

Sets can have infinite number of elements

Note that A = B if and only if

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Basic Set Operations

Complementation:
Union:
Also

Intersection:
Also
can also be written simply as AB.

Set Difference:
B\A is a set of that do not belong to A

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Venn Diagrams
Venn diagrams are very useful in set theory. They are
often used to portray relationships between sets.
Many identities can be read out simply by examining
Venn diagrams

For Example:

Let
Also let
And let
The Venn Diagram will be.

GSN3113: Probability and Stochastic Process UCP Fall 2015

4
B

A
1
2

5
6

Dr. Asim Baig

Venn Diagram

A\ B

GSN3113: Probability and Stochastic Process UCP Fall 2015

B\A

Dr. Asim Baig

An Example
Coffee Venn Diagram

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Some Basic Set Identities.

Idempotence:

Commutatively (Symmetry):
Similarly

Associativity:

Distributivity:

To avoid repeating the

word set multiple
times we call a set of
sets a
collection/class/family
of sets

De Morgan Laws:

Disjoint Sets:
Set A and B are said to be disjoint if and only if
represents empty set
In Probability we say that A and B are independent
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Mutually Exclusive
A collection of sets is said to be pairwise
disjoint or mutually exclusive if an only if
For Example sets A, B, C are Mutually
Exclusive if
A

GSN3113: Probability and Stochastic Process UCP Fall 2015

B
C

Dr. Asim Baig

Partition.
Given a set S, a
collection
of subsets of S is said to
be a partition of S if

For all

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Some more important

Concepts
Cardinality
The cardinality or size of a collection or set A, denoted
by |A|, is the number of elements of a collection
Cardinality can be finite or infinite
A finite set is a set that has a finite number of elements.
A set with infinite number of elements is called an infinite set.
Countable sets:
Empty set and finite sets are automatically countable.
An infinite set is countable if elements of A can be
enumerated or listed in sequence.
Singleton A singleton is a set with exactly on element.

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Relationship between Set Theory and

Probability Theory
Set Theory

Probability Theory

Set

Event

Universal Set

Sample Space / State Space()

Element

Outcome / State ()
Event

Language
A occurs
A does not occur
Either A or B occurs
Both A and B occur

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

What are the odds

CLASSICAL PROBABILITY

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Classical Probability.
Classical Probability is based upon the ratio of
the number of outcomes favorable to the
occurrence of the event of interest to the total
number of possible outcomes.
It was the first type of probability studied by
mathematicians such as Fermat and Pascal.
They are widely credited with starting the formal study
of probability.
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Classical Probability
Given a finite sample space , the classical probability of an
event A is

In this form probability is a faction such that

The Denominator represents the number of possible outcomes, and
The Numerator represents the number of outcomes in which the event of
interest occurs.

We can only calculate probability only if following assumption

are true
Finite -- The number of possible outcomes are finite
Equi-possibility: The outcomes have equal probability of occurrence.
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Equi-possibility.
Equi-possibility is also called the following in literature:

Equi-probable
Equi-possible
Equally Likely
Fair

The bases for identifying equi-possibility were often:

Physical Symmetry (e.g. a well-balanced dice, made of homogeneous material in
a cubical shape)
A balance of information or knowledge concerning the various possible
outcomes.

Equi-possibility is meaningful only for finite sample space,

therefore, Classical Probability (as a rule) can only work for finite
sample space.
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Examples
Suppose you have 9,999 red balls and 1 black ball.
What is the probability of getting a Black Ball.
Event A is getting a Black Ball
Sample Space is a set of all (10000) Balls
So the probability of event A is
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Another Example
In drawing a card from a deck, there are 52 equally likely outcomes, 13 of
which are diamonds. What is the probability of selecting a card with diamonds.

Event A is selecting a diamonds card

The sample space is the deck of cards

Then the probability of selecting a diamond card is

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Some Basic Properties of

Classical Probability
Here are some basic properties of Classical Probability that
will be useful for us in the rest of the course

this is derived directly from

Suppose and then

The probability of an event is equal to the sum of the probabilities of its
component outcomes because outcomes are mutually exclusive
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Another Example
Lets play with dice now
What is the probability of getting a 2 when we roll a dice
Event A is roll a 2
Sample Space is the set of all possible values that can
come up for a single throw of a dice
The probability of Event A occurring will then be

One dice is too simple . Lets make things

interesting and start working with two dice
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Two Dice
Assume that the two dice are fair and
independent
Then what is the probability of getting a 5 on a single
throw of two dice?

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Two Dice
What is the probability of getting a 5 with a single throw of two
dice.
Event A is getting a 5
Sample Space is all possible combination of values of the
two dice

Probability of getting a 5 is then

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Two Dice

Not as simple as it seem

Even a great mathematician like Leibniz was confused by the problem and initially
assumed that it was equally likely to throw as 11 or 12 with two dice.
Interested in working with dice
a bit more Check out these
sites with dice simulators
http://www.dicesimulator.com
And
http://www2.whidbey.net/ohmsm
ath/webwork/javascript/dice2r
ol.htm

GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig

Its not just two dice only.

Similar mistake was made by another mathematician when he claimed that

If a fair coin is tossed twice, the number of heads that turns up in those two tosses
can be 0, 1, or 2. Since there are three outcomes, the chances of each must be 1 in
3.

Can you find the mistake in the above statement???

Possible
Outcom
es

Probabilities

HH

HT

TH

TT

GSN3113: Probability and Stochastic Process UCP Fall 2015

Probabilities

Dr. Asim Baig

Probability and Gambling

In the world of gambling, probabilities are often expressed by
odds.
The statement that the odds are n:1 against an event means that
It is n time more likely that the event will not occur
Mathematically
this implies that
and

Note that ODDS here have nothing to do with even and odd numbers. They
represent what you will win in addition to getting you stake back should your
guess proves right.

Example: If I bet $1 on a horse with odds of 7:1

I get $7 + $1 = $8 if the horse wins the race
This means that the only way the bookmaker will break even in the long run is
if the probability of the horse winning is 1/8 (not 1/7)

Even ODDS mean 1:1 i.e.

You get $1 + $1 if you bet $1
The probability then is
GSN3113: Probability and Stochastic Process UCP Fall 2015

Dr. Asim Baig