Dharan MSC 2075 DP L8

6/6/2018
Dynamic Programming
System Engineering
MSc
MScininLand & WaterEngineering
Hydropower Engineering
Institute of Engineering
Dharan Campus Dharan
2075
Dr. Bhola Ghimire M.Sc. Land and Water Engineering System Emgineering 2075
Basic Introduction Dynamic Programming

• Introduction
• Recursive equations are used to solve a problem • Dynamic programming like the divide and conquer
in sequence method, solves problem by combining the solutions of
• These equations are fundamental to the dynamic sub problems
programming
• Objectives
• Divide and conquer method partition the problem
• To formulate recursive equations for a multistage
decision into independent sub problems, solves the sub
• process problems recursively and then combine their
• In a backward manner and solutions to solve the original problem.
• In a forward manner
Dr. Bhola Ghimire M.Sc. Land and Water Engineering System Emgineering 2075 Dr. Bhola Ghimire M.Sc. Land and Water Engineering System Emgineering 2075
Dynamic Programming Dynamic Programming
• Dynamic programming is applicable, when the sub‐ • Frequently, there is a polynomial number of sub‐
problems are NOT independent, that is when sub‐ problems, but they get repeated.
problems share sub sub‐problems.
• A dynamic programming algorithm solves every sub‐
• It is making a set of choices to arrive at optimal problem just once and then saves its answer in a table,
thereby avoiding the work of re‐computing the answer
solution. every time the sub‐problem is encountered
• If sub‐problems are not independent, we have to • So we end up having a polynomial time algorithm.

further divide the problem.
• Which is better, Dynamic Programming or Divide &
• In worst case, we may end‐up with an exponential conquer?
time algorithm.
6/6/2018
Optimization Problems 4 steps of Dynamic Programming

Algorithm
• Dynamic problem is typically applied to
1. Characterize the structure of an optimal solution.
Optimization Problems
2. Recursively define the value of an optimal solution.
• In optimization problems there can be many
possible solutions. Each solution has a value and 3. Compute the value of an optimal solution bottom‐up.
the task is to find the solution with the optimal (
Maximum or Minimum) value. 4. Construct an optimal solution from computed
• There can be several such solutions. information
Often only the value of the optimal solution is

required so step-4 is not necessary.
Recursive Definition
of the Fibonacci Numbers
The Fibonacci numbers are a series of numbers as follows:
Recursive ???
fib(1) = 1
fib(2) = 1
1, n <= 2
fib(3) = 2 fib(n) = fib(n-1) + fib(n-2), n > 2
fib(4) = 3
fib(5) = 5
...
fib(3) = 1 + 1 = 2
fib(4) = 2 + 1 = 3
fib(5) = 2 + 3 = 5
Fibonacci numbers
In mathematics, the Fibonacci numbers or Fibonacci sequence are
the numbers in the following integer sequence:
1,2,3,5,8,13,21,34,55,89,144,…….. or including 0,
By definition, the first two numbers in the Fibonacci sequence are 1
and 1, or 0 and 1, depending on the chosen starting point of the
sequence, and each subsequent number is the sum of the previous
two.
A tiling with squares In mathematical terms, the sequence Fn of Fibonacci numbers is
whose side lengths are defined by the recurrence relation
successive Fibonacci
numbers
with seed values
The Fibonacci spiral: an approximation of the golden spiral or

created by drawing circular arcs connecting the opposite
corners of squares in the Fibonacci tiling this one uses squares
of sizes 1, 1, 2, 3, 5, 8, 13, 21, and 34.
6/6/2018
Recursive Algorithm
The Fibonacci sequence is named after Fibonacci.

• Takes Exponential time,
Book by Fibonacci, 1202, named: Liber Abaci introduced the
sequence to Western European mathematics, although the
• Actual sub problems are polynomial (O(n)) but
sequence had been described earlier in Indian mathematics. they get repeated
• Sub problems are not INDEPENDENT.
By modern convention, the sequence begins either with
F0 = 0 or with F1 = 1. • Sub problems share sub‐sub problems.
The Liber Abaci began the sequence with F1 = 1. • We can solve it using Dynamic programming.
Benefit of Dynamic Programming Matrix Chain Multiplication
• Run an O(n) time loop, keep a temporary
One of the example of dynamic programming
variable to store the solution of sub‐problems
and then reuse them rather then recalculating
is an algorithm that solves the problem of
them.
• So by using dynamic programming we can
Matrix Chain Multiplication
solve a problem in polynomial time which
otherwise was solved in exponential time.
• We are given a sequence (chain) <A1, A2, A3, ...,An> , of n
matrices to be multiplied together. And we wish to
compute the product A1A2A3…An
• A product of matrices is fully parenthesized, if it is either a
single matrix or product of two fully parenthesized matrix
product, surrounded by parenthesis.
• As matrix multiplication is associative, so all
parenthesizations yield the same product/result.
6/6/2018
Matrix multiplication revisited
We can multiply two matrices A and B, only if the
number of columns of A is equal to the number of
rows of B.
If A is a p x q matrix, and B is a q x r matrix, the

resulting matrix C is p x r matrix.
So the time to compute C, is dominated by scalar
multiplications i.e. pqr
Thus computing the product according to the 1st
parenthesization is 10 times faster.
This clearly demonstrates the benefit of calculating the
optimum order before commencing the product
calculations.
(100x50)
Counting the number of

parenthesization.
• For a single matrix, we have only one parenthesization.
• The number of alternative parenthesization for a sequence
of n matrices is denoted by P( n).
• We can split a sequence of n matrices between the kth and

(k+1)st matrices for any k=1,2…n‐1, and then parenthesize
the two resulting subsequences independently.
6/6/2018
Computing the optimal costs
The idea of dynamic programming is, rather
than computing m (min number of scalar
multiplications) recursively, computing it bottom
up: A recursive computation takes exponential
time; a bottom‐up computation O(n3) time.
Elements of dynamic programming
When should we look for a dynamic
programming solution to a problem?
Two key ingredients?
– Optimal sub‐structure
– Overlapping sub‐problems
6/6/2018
Elements of dynamic programming
Optimal sub‐structure
A problem exhibits optimal sub‐structure, if an
optimal solution to a problem contains within
it optimal solutions to sub‐problems.
Overlapping Sub problems
Top‐down approach Vs Bottom‐Up approach
Bottom‐Up approach
More efficient because it takes advantage of the overlapping
sub‐problem property. There are only θ(n2) different sub‐
problems. Each problem solved exactly once.
Top‐down approach
Must repeatedly resolve each sub‐problem, each time it
reappears in the recursion.
6/6/2018
Dynamic a detail review Principle of optimality
DP is an enumerative technique developed by

Richard Bellman in 1953.
This technique is used to get the optimum solution to
a problem which can be Represented as a multistage
process. Let the optimal path for going from A to D be ABCD.
The entire DP Formulation is based on Bellman According to Bellman’s theorem, the optimal path
principle of Optimality. From B to D will be BCD and not BED. If the optimal
Path from B to D is BED, then the optimal path
from A to D will be ABED and not ABCD.
In a DP problem formulation, the dynamic

behavior of the system is expressed by using
ThreeTypes of variables
1. State variables: define the condition of the system.
eg. The amount of water stored in the reservoir
may represent its state.
If a problem has one state variable per stage, it is
called a one dimensional problem;
a multi-dimensional problem has more than one state
variable per stage.
Thus, the Optimization of operation of a system of two
or more reservoirs will have two or more state
variables.
Typical formulations
2. Stage variables: define the order in which events
Occur in the system. Most commonly, time is the • Recursive Equations
Stage variable. There must be a finite number of
Possible states at each stage. • Recursive equations are used to structure a
multistage decision problem as a sequential
3. Control variables: represent the controls applied at a
particular stage and transform the state of the system. process
for a reservoir operation problem, the release of water
from the reservoir is a typical control variable. • Each recursive equation represents a stage at
which a decision is required
6/6/2018
• A series of equations are successively solved,
each equation depending on the output values
of the previous equations
• A multistage problem is solved by breaking into
a number of single stage problems through
recursion
• Approached can be done in a backward manner
or in a forward manner
6/6/2018
• Example:
A total of 6 units of water is to be allocated

Optimally to three users. The allocation is made
in discrete steps of one unit ranging from 0 to
6. With the three users denoted as User 1,
User 2 and User 3 respectively, the returns
obtained from the users for a given allocation
are as follows:
The problem is to find allocations to the 3 users
such that the total return is maximized.
Return from
Amount of water User 1 User 2 User 3
Exercise 1:
allocated R1(x) R2(x) R3(x) Solve the following 4-user water allocation problem to
x maximize the total returns, using dynamic programming:
0 0 0 0 Water available for allocation = 60 units, to be allocated
1 5 5 7 in discrete units of 0,10, 20, .., 60. Returns from the four users
for a given allocation, are given in the table below:
2 8 6 12
Allocation Returns from
3 9 3 15 User 1 User 2 User 3 User 4
4 8 ‐4 16 0 0 0 -3 1
5 5 ‐15 15 10 3 4 3 1
6 0 ‐30 12 20 5 4 5 1
30 6 4 5 7
40 3 4 4 8
50 3 6 2 10
60 3 7 0 10
[ 10 10 10 30 Max Return> 17]
6/6/2018
Reservoir Operation Using DP

Release Policy:
Inflow (Qt) Sequence of releases, {Rt} ,
in a year, is called Reservoir
Storage (St, St+1) Release Policy
Storage Continuity Equation:

Release (Rt) St+1 = St + Qt - Rt
Starting with the last period, T, in the year, we
Benefit Function solve this problem with backward recursion.
Qt St+1 T
Max  Bt (St , Rt ) Stage 1:

St Rt t 1
n=1, and t=T,
Where, Bt (St , Rt ) is the net
benefit during period t for
given values of St and Rt, and
Dam T is the number of periods in
a year.
Stage 2:
n=2, and t=T-1,
Finally we have to back trace to the results
as earlier.
.
In general the recursive equation for any period t,
At last stage, t=1, n=T, S1 is said to be S’ , a known

value
Reservoir Operation problem using DP CAPACITY EXPANSION Problem

Inflows during four seasons to a reservoir with storage capacity of 4 units
are, respectively 2, 1, 3 and 2 units. Only discrete values 0, 1, 2, … are We may required to decide on investment on
considered for storage and release. Overflows from the reservoir are also
included in the release. Reservoir storage at the beginning of the year is 0 capacity expansion of w/s project?
units. Release from the reservoir during a season results in the following
benefits, which are same for all the four seasons. Obtain the reservoir
operation policy. [Assume one unit equal to one million m3]
May be each 5 years or any interval??
Releases Benefits
(Units)
0 -100
1 250
2 320
3 480
4 520
5 520
6 410
7 120 [ 1 1 3 2 > 1460]
6/6/2018
Term Definition..
Example- Capacity expansion

Ac city w/s project has to be expand for the storage
system from existing 100 units capacity to 200 units
capacity, in next 20 years of each 5 years revision in
expansion. The discounted present worth for additional
capacities are as follows:
Solution : see white board

6/6/2018
Shortest Route Problem Characteristics of DP

i. The problem can be divided into sub-problems (stages), with a policy
decision required at each stage.
ii. Each stage has a number of states associated with it.
iii. Decision at each stage converts the current state into another state
which is associated with the next stage.
iv. Every state is described by a set of variables called state variables.
v. Given the current stage, an optimal policy for remaining stages is
independent of the policy adopted in the previous stages.
vi. The solution procedure starts by finding an optimal policy for each
state of the last stage.
vii. A function (recursive) equation is available which identifies the optimal
policy for each state with n stages remaining, given the optimal policy
for each state with (n-1) stages remaining.
viii. Using the functional equation, the solution procedure moves backward
stage by stage, each time finding the optimal policy when starting at
In Figure above, which route is shortest the initial stage. Or moves forward stage by stage and associate the
starting from A to reach at E? further stages as earlier.
Thank You!!!
Dr. Bhola Ghimire M.Sc. Land and Water Engineering System Emgineering 2075 69

Dharan MSC 2075 DP L8

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dharan MSC 2075 DP L8

Uploaded by

Copyright:

Available Formats

6/6/2018

Dr. Bhola Ghimire M.Sc. Land and Water Engineering System Emgineering 2075

Basic Introduction Dynamic Programming

Dynamic Programming Dynamic Programming

• If sub‐problems are not independent, we have to • So we end up having a polynomial time algorithm.

Optimization Problems 4 steps of Dynamic Programming

Often only the value of the optimal solution is

The Fibonacci spiral: an approximation of the golden spiral or

The Fibonacci sequence is named after Fibonacci.

Benefit of Dynamic Programming Matrix Chain Multiplication

Matrix Chain Multiplication

Matrix multiplication revisited

If A is a p x q matrix, and B is a q x r matrix, the

Matrix Chain Multiplication

Counting the number of

• We can split a sequence of n matrices between the kth and

Computing the optimal costs

Elements of dynamic programming

Elements of dynamic programming

Overlapping Sub problems

Dynamic a detail review Principle of optimality

DP is an enumerative technique developed by

In a DP problem formulation, the dynamic

A total of 6 units of water is to be allocated

Reservoir Operation Using DP

Storage Continuity Equation:

Max  Bt (St , Rt ) Stage 1:

At last stage, t=1, n=T, S1 is said to be S’ , a known

Reservoir Operation problem using DP CAPACITY EXPANSION Problem

Example- Capacity expansion

Solution : see white board

Shortest Route Problem Characteristics of DP

Dr. Bhola Ghimire M.Sc. Land and Water Engineering System Emgineering 2075 69

You might also like