Data Structures

Dr Ahmed Rafat Abas

Computer Science Dept, Faculty of Computer and
Information, Zagazig University
arabas@zu.edu.eg
http://www.arsaliem.faculty.zu.edu.eg/
Algorithm Analysis

Chapter 3
3.1 Introduction
Algorithm analysis is a method that measures the
efficiency of:
an algorithm, or
its implementation as a program,
when the input size becomes large.

This method analyzes:

the time required for an algorithm, or its
implementation as a program, and
the storage space required for a data structure.
Estimating an algorithm's performance can be carried
out through estimating the number of basic operations
required by the algorithm to process an input of a certain
size.

The term size is the number of inputs processed.

example: when comparing sorting algorithms, the size of
the problem is measured by the number of records to be
sorted.

A basic operation requires a time to be completed, which

does not depend on the values of the operands of this
basic operation.
example: adding two integers is a basic operation.
c

Example: compute the running time of the

following code:
Sum=0;
For (i=1; i<=n; i++)
For (j=1; j<=n; j++)
Sum++;

Assuming the time required for incrementing the

sum variable is c, the running time is computed
as: T (n) c cn2
This equation describes the growth rate of the
running time of the previous code.
The concept of growth rate is important
as it allows us to compare the running time
of two algorithms without actually writing
two programs and running them on the
same computer.
A graph illustrating the growth rates for five equations.
The horizontal axis represents input size.
The vertical axis can represent time, space, or any
other measure of cost.
 Slow

2n Exponential growth rate

2n 2 Quadratic growth rate

T (n) 5nlog n Logarithmic growth rate

20n Linear growth rate

10n Linear growth rate

Fast

Ordering of the running time corresponding to

five growth rate equations from slow to fast.
3.2 Best, Worst and Average
Cases
The best case for an algorithm is the case corresponding
to minimum running time for this algorithm.

The worst case for an algorithm is the case

corresponding to maximum running time for this
algorithm.

The average case for an algorithm is the case

corresponding to running time equal to the average of
the best and the worst cases running time.

In practice, it is preferred to deal with a worst case

analysis of an algorithm.
3.3 Asymptotic Analysis
3.3.1 Upper Bounds
The upper bound for the running time of an algorithm is
the upper bound or the highest growth rate the algorithm
can have.

The upper bounds are measured on the best case,

average-case or worst case inputs.
example: - if the growth rate in the worst case for an
algorithm is T (n) cn ,
- then this algorithm has an upper bound to its growth
rate of nin the worst case.
-This phrase can be rewritten using a special notation
called the big-oh notation as follows: this algorithm is in
O(n), in the worst case.
3.3.2 Lower Bounds
The lower bound for the running time of an algorithm is
the least amount of time required by this algorithm for
some class of input.

This is a measure of the growth rate of this algorithm.

The input can be the worst, average, or best-case of
size.
example: - if the lower bound of the running time of an
algorithm in the worst case isT (n) cn ,
- then this algorithm has a lower bound to its growth rate
of nin the worst case.
- Alternatively, we can use the big notation as follows:
this algorithm is in (n) , in the worst case.
3.3.3 -Notation
When the upper and lower bounds are the same
within a constant factor, this case can be
indicated by the use of (big-theta) notation.
example: an algorithm is said to be (h(n)) if it is
in (h(n)) and it is in (h(n))

Note that the word "in" is dropped for notation,

since there is a strict equality for two equations
with the same .
3.3.4 Simplifying Rules
In determining , , from the
running time equation, the following
rules are considered:
1. if f (n) is in ( g (n)) and
g (n) is in (h(n)) , then
f (n) is in (h(n)) .
This rule holds true for and
notations.
2. if is in (Kg (n)) for any constant
f (n)
K 0 , then
f (n) is in ( g (n)) .
This rule holds true for and
notations.
3. if f1 (n) is in ( g (n)) and
1

f 2 (n) is in ( g 2 (n)) , then

f1 (n) f 2 (n) is in (max(g (n), g (n))) .
1 2

This rule holds true for and

notations.
4. if f1 (n) is in ( g1 (n)) and
f 2 (n) is in ( g 2 (n)) , then
f1 (n) f 2 (n) is in ( g (n) g (n)) .
1 2

This rule holds true for and

notations.
Example:
If T (n) 3n 2n , then
4 2
T (n) is in
(n ) .
4
3.3.5 Summations and Closed
Form Solutions
Summation = closed form solution
n

f (i) f (1) f (2) ... f (n)

i 1
n
n(n 1) n 3
2
n

2 n 3n
i 1
i
2 ,
i 1
i
2

6 ,
log n
1
n n log n ,
i 1

i 0
a i

1 a , for 0 a 1
 n 1

, for a 1
n
1 1 n
a 1

i 1 2
i
1 n
2 ,
i 0
a
i

a 1
n2
n

n
i

n 1
i
2 n , 2 i
2 1
i 1 2 2 i 0
log n

2
i 0
i
2 log n 1
1 2n 1
n
1
Harmonic series n ,
i 1 i

log e n n 1 log e n
3.4 Calculating The Running
Time for a Program

Example:
a=b;
then T (n) is (1) .
Example:
sum=0;
for (i=1; i<=n; i++)
sum += n;
then T (n) is (c cn) , which is
simply (n) .
Example:
Sum = 0;
for (j=1; j<=n; j++)
for (i=1; i<=j; i++)
sum ++;
for (k=0; k<n; k++)
A[k] = k;
n
(c cj cn)
Then T (n) is j 1
, which can
n(n 1)
be simplified to ( c c cn) that is
2

(n ) .
2
Example:
Compare the asymptotic analysis for the following
two code segments:
sum1=0; sum2=0;
for (i=1;i<=n;i++) for (i=1;i<=n;i++)
for (j=1;j<=n;j++) for (j=1;j<=i;j++)
sum1++; sum2++;
then
1T ( n) is ( c1 c1 n 2
) , which is simply ( n 2
).
n
n(n 1)
T2 (n) is (c2 ic2 ) , which is ( c2 c 2 ) that
i 1 2
is simply ( n 2
).
Therefore, both code segments cost ( n 2
) , but the
second requires half the time of the first when both of
them are run on the same computer.
Example:
Compare the running time of the following two code
segments:
sum1=0; sum2=0;
for (k=1; k<=n; k*=2) for (k=1; k<=n; k*=2)
for (j=1; j<=n; j++) for (j=1; j<=k; j++)
sum1++; sum2++;
then,
log n

T1 (n) is (c1 c1 n) , which is (c1 c1 (n n log n))

i 0

that is (n log n) .
log n
2
i
2
T ( n) is ( c 2 c 2 ) , which is (c2 c2 (2n 1)) that
i 0

is (n) .
3.5 Space Bounds
Storage space is also an important factor
for the program efficiency.

The analysis techniques used to measure

space requirements are similar to those
used to measure time requirements.
Example:
The space requirements for an
array of size n integers is (n) .
There are two important principles of
algorithm design:
The space/time tradeoff
this principle says that one can often
achieve a reduction in time if one is
willing to sacrifice space or vice versa.
The disk-based space/time tradeoff
this principle states that the smaller you
can make your disk storage
requirements, the faster your program
will run.