Dynamic Programming: Part 1 Classical Algorithms

Wilan Wong wilanw@gmail.com

Contents
1 Introduction 2 Fibonacci Sequence 3 The Knapsack Problem 3.1 3.2 3.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Modelling a Dynamic Programming Algorithm . . . . . . . . . . Other Knapsack variants . . . . . . . . . . . . . . . . . . . . . . . 3 3 8 8 10 15 18 18 23 25 25 25 26 27 28

4 Edit Distance 4.1 4.2 Modelling and Implementation . . . . . . . . . . . . . . . . . . . Performance Dierences . . . . . . . . . . . . . . . . . . . . . . .

5 Overview 6 Problems 6.1 6.2 6.3 6.4 Problem 1: Longest Increasing Subsequence . . . . . . . . . . . . Problem 2: Binomial Coecients . . . . . . . . . . . . . . . . . . Problem 3: Longest Common Subsequence . . . . . . . . . . . . . Problem 4: Handshaking Problem . . . . . . . . . . . . . . . . .

Introduction

Dynamic programming is one of the major techniques used within combinatorial optimisation. Combinatorial optimisation algorithms solve problems that are believed to be hard in general, by searching the large solution space of these problems. However, due to the large solution space many problems become too unwieldy to be solve with brute force based algorithms, so combinatorial optimisation algorithms achieve better performance by reducing the search space by discarding portions of the solution space that can be deemed non-optimal, these algorithms also usually explore the space eciently so they dont recalculate identical problems. Simply put, dynamic programming is a method to exploit inherit properties of a problem, so the algorithm runs much faster than simpler straightforward brute-force algorithms. The basis behind dynamic programming is whats known as a functional equation. You can think of this as some sort of recurrence relation which relates the problem we are trying to solve with smaller subproblems of the same problem. This in eect, means that dynamic programming is recursive in nature a solid understanding of recursion is required to understand dynamic programming even in its most basic form. There consists of many deep mathematical theories behind dynamic programming, so we will only consider the fundamentals whose coverage should be sucient enough for you to develop a basic dynamic programming formulation skill. We now cover basic recursion skills, show its limitations when used naively and consequently provide a simple method to translate recursion to top-down dynamic programming.

Fibonacci Sequence

Let us consider a trivial problem that you probably already know of: Fibonacci numbers. If you havent seen Fibonacci numbers before, basically its a sequence of numbers where the next term is dened by the sum of the previous two terms. You start o with the rst two terms being 1, so the sequence looks like: 1, 1, 2, 3, 5, 8, 13, 21 etc. The goal of this problem is to compute the n -th Fibonacci number, so the rst Fibonacci number would be 1, the third would be 2, the sixth would be 8 and so on. How do we formulate a recursive algorithm for this problem? Lets assume we know all Fibonacci numbers up to the (n-1 )th term, now we ask ourselves how do we obtain the n -th term. If we knew all the Fibonacci numbers up to the (n-1 )th one then by the denition of a Fibonacci sequence: Fn = Fn1 + Fn2 we can calculate the next one. This itself forms the basis of the recursion, it denes the current problem (Fn ) in terms of two smaller subproblems (Fn1 and Fn2 ) in which they are related by the addition of the two subproblems. As you may know from recursion it is crucial to have a base case, and the 3

recursion itself should move closer and closer to this base case otherwise it might not terminate for certain inputs. So our second task after dening the recurrence relation (Fn = Fn1 + Fn2 ) is to determine suitable stopping conditions (or base cases). Since the recurrence relation relies on the previous two terms we must provide at least 2 consecutive base cases otherwise the relation would be undened. If we look back at the sequence we see that the rst two terms are just 1. So we can dene F0 = F1 = 1 as the base case. Looking back we can formally dene our functional or recursive equation as: 1 F (n 1) + F (n 2) if n = 0, 1 otherwise

F (n) =

Now, we simply just translate the above to a programming language. We will be demonstrating our algorithms in C++ for this tutorial, most readers should have little diculty transferring it to their preferred language of choice. See Listing 1 for the implementation. Listing 1: Fibonacci Naive Recursion Approach
1 #include < i o s t r e a m > 2 3 using namespace s t d ; 4 5 long long f i b ( i n t n ) { 6 i f ( n <= 1 ) return 1 ; 7 return f i b ( n1)+ f i b ( n 2); 8 } 9 10 i n t main ( ) { 11 int n ; 12 while ( c i n >> n ) c o u t << f i b ( n ) << "\n" ; 13 return 0 ; 14 }

The rst thing to note if you have compiled and ran this algorithm, is that it gets very slow for even modest sized inputs. One thing that may come to mind is that recursion is just plain slow or is it? Certainly, there is additional overhead for recursion due to stack modications for function calls (i.e. pushing and popping o local and function parameters on the stack) however it certainly shouldnt have an exponential impact. If we draw out the recursion tree of the Fibonacci numbers, we yield the source of our problem. The problem lies with multiple calls to the same functions that we have already calculated previously the diagram below illustrates this better.

Figure 1: Fibonacci Call Tree As you can see above when we compute the 4th Fibonacci number using our naive recursion, we call F(2) twice. For such a small recursion tree it may not matter - but for larger numbers signicant re-calculation work is done. If you are not convinced from this small tree, try sketching a recursion tree for the F(10) and youll quickly see the point. This property is called overlapping subproblems which as the name implies, refers to when our recursion keeps recalling the same subproblems over and over hence the term overlapping. Now another property the Fibonacci problem exhibits is referred to as optimal substructure. This means that an optimal solution can be constructed from solutions to its subproblems. Referring back to Figure 1, if the two calls to F(2) had dierent results each time we called them then this is when the problem does not exhibit optimal substructure. However, for all our cases we can be sure that the 2nd Fibonacci number will not change after we have calculated it by denition of the Fibonacci sequence, so we can eectively and safely assume it for our algorithm. In these situations, the problem is said to exhibit optimal substructure. The two properties: optimal substructure and overlapping subproblems are required for a dynamic programming algorithm to work both correctly and eciently. If the problem does not have any overlapping subproblems, then just using the simple recursion would be as ecient as we could get because we arent doing any extra work by re-calculating subproblems again and again so using dynamic programming only adds additional overhead. If the problem does not have optimal substructure, then our dynamic programming algorithm may run quickly but will produce incorrect results because it incorrectly assumes false information the further examples would illustrate this much clearer. So thats a whole heap of talk about the properties required for a dynamic programming algorithm but how do we actually formulate and code it? A simple way is to note that because our naive Fibonacci recursive algorithm recalculates the same subproblems over and over, why not make a simple lookup array to check if we have already calculated that value before we go diving 5

further into the recursion tree? So this logically works by having a special value that denotes that we have not calculated that problem before (usually negative numbers but may change depending on the nature of the problem), then for the recursive function, we check if the current value in the look-up array corresponding to the function input is equal to the special value. If it is, then we know that it still hasnt been calculated, otherwise if it isnt, then we already know the result of that input and we can just return the value of the look-up array in that specic index. It may sound complex, but augmenting it is relatively easy and is usually universal across many dynamic programming algorithms. (See Listing 2 for implementation) Listing 2: Fibonacci Memoization Approach
1 #include < i o s t r e a m > 2 3 using namespace s t d ; 4 5 #define MAX TERMS 100 6 7 long long memo [MAX TERMS+ 1 ] ; 8 9 long long f i b ( i n t n ) { 10 i f ( n <= 1 ) return 1 ; 11 i f (memo [ n ] != 1) return memo [ n ] ; 12 return memo [ n ] = f i b ( n 1) + f i b ( n 2); 13 } 14 15 i n t main ( ) { 16 int n ; 17 memset (memo, 1 , s i z e o f (memo ) ) ; 18 while ( c i n >> n && n <= MAX TERMS) c o u t << f i b ( n ) << "\n" ; 19 return 0 ; 20 }

You should compare the running times between our rst attempt and after we applied a caching mechanism to the recursion. In fact, theres an exponential dierence in running time between the two algorithms even though they are fundamentally similar. Also note, that you will need to use arbitrary-precision integers very soon above the Fibonacci term of 100 because the numbers growth quickly and exceed the 64-bit integer data-type limit. You may have coded another iterative algorithm for the Fibonacci sequence before in an introduction to programming course (see Listing 3), which is to keep track of the last two terms and keep generating the next term based on a for-loop or a similar construct. In eect, that method is dynamic programming it diers in the order of execution because it approaches the problem bottom-up, i.e. it solves the simplest cases rst and gradually builds up to larger cases. The 6

Listing 3: Fibonacci Iterative Variant

1 #include < i o s t r e a m > 2 3 using namespace s t d ; 4 5 i n t main ( ) { 6 int n ; 7 while ( c i n >> n ) { 8 i f ( n <= 1 ) { 9 c o u t << " 1\ n" ; 10 continue ; 11 } 12 long long prev = 1 , prev2 = 1 ; 13 f o r ( i n t i = 2 ; i <= n ; i ++) { 14 long long temp = prev ; 15 prev += prev2 ; 16 prev2 = temp ; 17 } 18 c o u t << prev << "\n" ; 19 } 20 return 0 ; 21 }

recursion + look-up method is called memoization and is also known as top-down dynamic programming, its called top-down because it seeks to solve larger instances of the problem rst and gradually moves down until it hits the base cases (which are usually the simplest cases of the problem). Both methods have similar run-time performance but there are subtle dierences to note. Bottomup is usually considered more ecient because it doesnt have the stack overhead of recursive algorithms, however memoization does have an advantage in that it only calculates the subproblems that it needs whereas bottom-up needs to solve all of the subproblems up to the current input. Its somewhat easier to learn dynamic programming using the top-down approach because when you use bottom-up dynamic programming you have to ensure the order in which you calculate the subproblems is correct (i.e. you have to make sure you have already calculated all the required subproblems when you reach a given problem) otherwise the whole dynamic programming approach breaks up. Top-down dynamic programming achieves this implicitly through the recursive call structure so you only need to worry about setting up the recursive algorithm. That being said, all top-down dynamic programming algorithms can be converted to bottom-up and vice versa. Nevertheless, we will cover bottom-up dynamic programming in our next problem which is slightly more complicated than the one we have seen so far.

3
3.1

The Knapsack Problem

Introduction

Lets consider a more complex example than the Fibonacci problem discussed previously. The next problem we will be looking at is called the Knapsack problem. Its a classic Computer Science problem which entails the following: Youre a thief that has successfully inltrated inside a bank. However to your surprise there are only items of questionable value as opposed to money. You bought a bag that can only hold a certain weight (W), you want to pack items into the bag so that you collect the best possible value of items whilst not packing more than the limit of W. Each item has a weight and a value. Your task would be to return the highest value you can get from this bank robbery. An intuitive approach might be to pack items by the best value per weight ratio until the bag is full or cannot hold anymore items. This is called a greedy algorithm, in which each decision stage you choose the most locally optimal choice (i.e. in this case, the best value/weight ratio that hasnt been taken and also ts in the bag). However, without a correctness proof we wouldnt be sure it would work for all cases. In fact, this greedy strategy does not work for the knapsack problem in general. How do we show this? We simply need to nd a counter-example. Lets consider an example, where the greedy strategy does indeed get the optimal solution: Bag Weight (W) = 10 kgs Item Configurations (Weight, Value) Pairs: (5,5), (3,2), (1,0), (10,6), (8,5) Using the greedy strategy, we rst calculate all the value per weight ratios. These are listed as: Item Configuration Ratios: 1.0, 0.66, 0.0, 0.6, 0.625 respectively. Now we begin to pack the items. Keep in mind, there are dierent variations of the Knapsack problem, one variation is that there is an unlimited number of items of each type, and another variation consists of only allowing a limited amount of each type of item. We will consider the unlimited variation rst and later discuss on the limited version. Now back to the greedy strategy, we see that the item of 5kg and value of $5 is the best in terms of ratio. We pack one of them inside our bag, we have 5kgs remaining and we repeat the same process (huge hint for recursion). Again we choose the same item type 8

because it can still t, now we have 0kg remaining so we halt. In the end, we chose 2 items of (5,5) which gave us a total value of $10. Convince yourself this is the best you can get from any combination of the items. Does this mean that our greedy strategy works? The simple answer, no! Lets consider an example that breaks the intuitive strategy, Bag Weight (W) = 13 kgs Item Configurations (Weight, Value) Pairs: (10,8), (8,6), (5,3), (10,6), (1,0) Item Configuration Ratios: 0.8, 0.75, 0.6, 0.6, 0.0 If we repeat the same process again, we end up choosing 1 x (10,8) because it has the highest ratio. So we are left with 3kg, however none of the items t (except for 3 x (1,0) which gives a value of $0). So the best value we can get from our greedy approach is $8. However, you can observe that you can get $9 of value from a combination of (8,6) and (5,3). Hence, this works as a counter-example against our greedy strategy and hence the greedy algorithm isnt correct. So how else can we approach this problem? One sure way to get a correct answer would be to brute force every single item conguration, check if it ts inside the bag and update it if its better than the current maximum. However, its horribly inecient to do it this way. A very naive approach based on this would be to generate subsets of items and test each one, however as you may know the number of subsets in a set is exponential (2n ). In fact, this problem is classied as NP-hard which means the best way we currently know how to do it is in exponential time (without getting too rigorous). However, dynamic programming provides a pseudo-polynomial time algorithm that correctly solves the Knapsack problem. Its important to distinguish between a pseudo-polynomial time algorithm and a polynomial algorithm. Loosely speaking, a polynomial algorithm is one where its time complexity is bounded by a constant power that isnt related to the input numerical values (not sizes) in any way examples would be O(n2 ), O(n3 ), O(n100 ). A pseudo-polynomial time algorithm grows in proportion to the inputs numerical value, so for example O(nm) would be a pseudo-polynomial time complexity if either n and m are related to the inputs numercial values. For example, if an algorithm runs in O(nm) where m is some constant and n varied with the largest input value, then the algorithm would be pseudo-polynomial. However, if the input values remain fairly small then the overall algorithm is still ecient. A dynamic programming Knapsack algorithm belongs to such a class as we will see shortly.

3.2

Modelling a Dynamic Programming Algorithm

However how do we approach this problem in a dynamic programming way? There are a few basic general guidelines used when formulating any dynamic programming algorithm: 1. Specify the state 2. Formulate the functional/recursive equation in terms of the state 3. Solve/code the solution using Step 2 4. Backtrack the solution (if necessary) Step 1: Specifying the state A state is simply information that we need in each step of the recursion to optimally decide and solve the problem. In the Fibonacci example, the state was simply the term number this alone allowed us to decide what the subproblems were and if they were base cases or not. For this example, it isnt as easy to come up with. In fact, for most dynamic programming algorithms coming up with a suitable and ecient state is the hardest part. For the Knapsack problem, when we try to look for the state a good indicator is what the recursion function parameters would look like. For simple problems, you can solely base your state on these, however for more complex problems where the state space can be huge you will need to do some fancy tricks and optimisations to ensure the state is as condensed as possible whilst still representing the problem in a correct fashion. So ask yourself, what are the common links between a specic problem and its subproblems? The knapsack subproblems were simply ones where the weight remaining was smaller. All the other factors remained the same (the available choice of items). Can we base our state solely on the weight? When you ask yourself this, just think if someone gave you a single parameter (the weight) and a list of possible items, could you decide the optimal decision to choose assuming you know the optimal values of all the weights beneath it? The answer to this is, yes! Thinking recursively is usually the best strategy, for example, if we were given a weight W to ll in the bag and we knew the best values we could get from weight 0 up to W 1, then a simple method would be to iterate through each of the lesser weights and try adding the best item to it so it can exactly ll weight W . We then choose the greatest value of these congurations. Convince yourself this is sucient by drawing some examples this is one of the two main properties of dynamic programming: optimal substructure. Step 2: Formulating the recursive equation Now we have a state which in this case is just weight remaining in the bag. We need to build a recursive relationship between states, i.e. link up a state to 10

another state. Usually this is obvious because it works in one direction: towards the base case. The base case for our problem would simply be the weight of 0. We realistically assume you cannot t anything in a bag that cant store anything hence the value of a bag weight of 0 is just simply 0. Since we are working towards this goal, we want recursive calls to generate successively lower weights. To determine a relationship between the states we take note that, we can add a single item at a time since order does not matter. So we can relate a lower weight state by adding items so it becomes exactly W . To determine which lower weight states are relevant, we just iterate through the item list of weights and subtract it from W , i.e. W Wi (where Wi is the weight of item i). If this value is less than 0 obviously we dont want it because that would violate the weight constraints (you are storing a heavier item than what could t in the remaining bag). So we simply choose the best out of these item additions. Hence the recurrence/functional equation may look like: max {F (w wi ) + vi } 0 if w wi 0, i in items if w = 0

F (w) =

What do we initialise F (w) before we begin comparisons? One option would be to set it to 0 however if we were to do this then we need to loop through all weight values at the end and choose the maximum. Another option would be to set F (w) to be equal to F (w 1) initially this makes sense because if an item conguration can t in w 1 then it should be able to t in w, doing it this way saves us having to loop through all the weight values at the end. Either option produces the correct output, so its just a matter of taste. Another alternative would be to add a dummy item with weight 1 and value 0 - this implicitly sets F (w) to be at least as much as F (w 1). So for step 2, we have the slightly modied recursive denition:

F (w) =

0 if w = 0 max {F (w wi ) + vi , F (w 1)} if w wi 0, i in items

The recursion implies that F (w) should be set to F (w 1) value without the need to fulll the w wi 0 requirement. Step 3: Coding the recursion equation Coding the recursive algorithm is usually fairly straightforward and simple. A C++ memoized solution is shown in Listing 4. If you look at the source code and compare it with our original recurrence formulation in step 2, you will see there are a lot of similarities in the recursive function f (). For non-C++ users, we have used a pair class which is basically a 11

Listing 4: Knapsack Top-Down Dynamic Programming

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 #include < i o s t r e a m > #include < v e c t o r > #include < u t i l i t y > #define MAX WEIGHT 1000 using namespace s t d ; i n t memo [MAX WEIGHT+ 1 ] ; v e c t o r <p a i r <int , int > > data ;

// ( w e i g h t , v a l u e ) p a i r

i n t f ( i n t w) { i f (w == 0 ) return 0 ; i f (memo [ w ] != 1) return memo [ w ] ; i n t v a l = f (w 1); f o r ( i n t i = 0 ; i < data . s i z e ( ) ; i ++) { i f ( data [ i ] . f i r s t <= w) { v a l = max( v a l , f (wdata [ i ] . f i r s t )+ data [ i ] . s e c o n d ) ; } } return memo [ w ] = v a l ; } i n t main ( ) { i n t weight , v a l u e ; i n t bagWeight ; c i n >> bagWeight ; i f ( bagWeight > MAX WEIGHT) return 1 ; while ( c i n >> w e i g h t >> v a l u e ) { data . push back ( m a k e p a i r ( weight , v a l u e ) ) ; } memset (memo, 1 , s i z e o f (memo ) ) ; c o u t << " Optimal value : " << f ( bagWeight ) << "\n" ; return 0 ; }

simple structure consisting of two members (both integers in our case) that serve to hold the weight and value of each item. We have also used the vector class which is basically a memory managed dynamic array, it should be intutitive to follow even if you have no knowledge of C++. The program also requires a structured input, taking in the bag weight as the rst integer to be read, follow by pairs of items (terminated by EOF). Note that if we take out our caching mechanism (memoization principle), we are left with just a simple recursive program (which is also inecent test this out yourself for large inputs).

12

You can see from the implementation above, there is a limitation of dynamic programming. What happens if the weight of 1 item was as large as 1 billion? We would have to make a caching array of 1 billion integers representing each of the weights that are possible. Now we have a memory limit situation - this algorithm doesnt scale well to large value inputs! We also have an execution time problem, in the worst case we have to recurse through 1 billion f () recursive functions (which will denitely overow the stack before it nishes its calculations). The point is that dynamic programming isnt a silver bullet for hard problems it only runs well under limited conditions. So you should denitely check whether the state space is too large before you begin formulating/coding the problem! Step 4: Backtrack the solutions Its all nice and good to know the best value we can get from the knapsack problem. However, the thief still scratches his head because although he knows the best value he can come out with, what items should he actually choose to obtain the optimal value? Its important to realise whether you need this step or not, if you do then you may need to take additional measures when you code the recursive algorithm. After the memoization has nished, we know have an array that details the optimal value for many dierent weights we need to use this (and usually some other bookkeeping information) to backtrack the items we have used to get there. How do we do this? At the moment it contains neither markers nor any bookkeeping to allow us to deduce which items where added at each stage of the dynamic programming process. So we need to change our implementation to do this. What markings do we need? We need some way to know which subproblem produced the best result for a particular weight w, and we need to know which item was added on top of that subproblem. To no surprise, this backtracking is also recursive in nature, you start o at the weight W (the bag weight), assume we know which item was added and which was the optimal subproblem, then we print out the item that was added and recurse to the optimal subproblem and repeat the process until we reach weight 0. The source code below demonstrates this approach using an extra integer auxiliary array that keep track of which item was added (the index of the item) we can directly deduce the weight of the subproblem since we know the weights of each item, we just need to subtract the item weight o the current weight. You can store it in an array instead of printing it, or do other processing its entirely up to you. See Listing 5 and 6 for implementation details.

13

Listing 5: Knapsack Top-Down Dynamic Programming with Backtracking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 #include < i o s t r e a m > #include < v e c t o r > #include < u t i l i t y > #define MAX WEIGHT 1000 using namespace s t d ; i n t memo [MAX WEIGHT+ 1 ] ; v e c t o r <p a i r <int , int > > data ; // ( w e i g h t , v a l u e ) p a i r i n t i n d e x T a b l e [MAX WEIGHT+ 1 ] ; // used f o r b a c k t r a c k i n g v e c t o r <int > itemsUsed ; // used f o r b a c k t r a c k i n g i n t f ( i n t w) { i f (w == 0 ) return 0 ; i f (memo [ w ] != 1) return memo [ w ] ; i n t v a l = f (w 1); f o r ( i n t i = 0 ; i < data . s i z e ( ) ; i ++) { i f ( data [ i ] . f i r s t <= w) { i n t k = f (wdata [ i ] . f i r s t )+ data [ i ] . s e c o n d ; i f ( val < k) { val = k ; i n d e x T a b l e [ w ] = i +1; // b o o k k e e p i n g i n f o r m a t i o n } } } return memo [ w ] = v a l ; } void b a c k t r a c k ( i n t w) { i f (w == 0 ) return ; itemsUsed . push back ( i n d e x T a b l e [ w ] ) ; b a c k t r a c k (w data [ i n d e x T a b l e [ w] 1 ] . f i r s t ) ; } i n t main ( ) { i n t weight , v a l u e ; i n t bagWeight ; c i n >> bagWeight ; i f ( bagWeight > MAX WEIGHT) return 1 ; while ( c i n >> w e i g h t >> v a l u e ) { data . push back ( m a k e p a i r ( weight , v a l u e ) ) ; c o u t << " Item : " << data . s i z e ( ) << " Weight : " << w e i g h t << " Value : " << v a l u e << "\n" ; }

14

Listing 6: Knapsack Top-Down Dynamic Programming with Backtracking (continued)

46 47 48 49 50 51 52 53 54 memset (memo, 1 , s i z e o f (memo ) ) ; c o u t << " Optimal value : " << f ( bagWeight ) << "\n" ; b a c k t r a c k ( bagWeight ) ; c o u t << " Items Used :\ n" ; f o r ( i n t i = 0 ; i < itemsUsed . s i z e ( ) ; i ++) { c o u t << " Item " << itemsUsed [ i ] << "\n" ; } return 0 ; }

3.3

Other Knapsack variants

Now we have a dynamic programming algorithm that completely solves the knapsack problem. Or have we? Lets consider another variant where the items cannot be re-used. How do we keep track of which items have been used in a concise manner? Informally, many people call the knapsack variant we just completed a 1 dimensional dynamic programming algorithm. This is because the state lies in a 1D state array (the weight). The variant we consider now is in fact, a 2 dimensional dynamic programming algorithm. This should serve as a hint in what the state would be! If you havent guessed it, the state for this variant is (weight, item). How does this work? Instead of keeping track whether or not we have used an item or not, we implicitly enforce the rule by iteratively considering a growing set of available items. For example, if we were given 8 items to pack, we then start o by considering the rst item. Calculate the optimal values for each of the weights for this item. Then we expand to the second item, we use the values for the rst item to decide optimal values for each of the weights including this item. By including this item, we really mean introducing it to the decision space we can actually choose to reject this item if it proves non-optimal. More formally we can set the state to be equal to: Let S(W,I) = the optimal value of a bag of weight W using item 1 up to item I. We build the recurrence relation by using the binary (two) decisions we have when we include an item do we include it or do we reject it? If we include the item i then we eectively consume wi worth of weight, if we choose not to include the item then we dont consume any weight. However we need to consider the base cases for the recurrence relation. We can re-use our previous base case where any w = 0 has a value of 0. Also if we choose to use 0 items or have 0 items 15

left to choose from, then we can assign a value of 0. Hence F (w, 0) = F (0, i) = 0 are the base cases for the recursion. The resulting recursive denition is as follows:

F (w, i) = max

{F (w wi , i 1) + vi } F (w, i 1)

if w wi 0, i in items

F (0, i) = F (w, 0) = 0 (base cases) If youre still unsure then try to think recursively. Dynamic programming is especially hard in the beginning but the underlying principles behind it are really just recursion. Implementation is usually the easiest part because its just merely translating the denition into code. You can do it top-down (memoization) or you could do it with a bottom-up approach. For this variant, we will implement it bottom-up it will really look like magic. The top-down approach will be an exercise for the reader which shouldnt be too hard to implement. So how do we go about implementing it bottom-up? What are the dierences between this and the top-down approach? The major dierence is that the order in which we compute the algorithm is important and crucial. However, for simple examples such as this the order is already implied by the recurrence relation. How do we deduce the order? What is the order? Imagine a double nested for loop going through a 2 dimensional matrix, you have two integer variables keeping track of the two indexes. The order is simply the order in which the coordinates (i.e. the two index pairs) are visited. Of course, its more abstract in dynamic programming but the principle is the same. To deduce the order, we look at the recurrence relation and see what a particular state depends on. This shows that (w, i) depends on (w wi , i 1) and (w, i 1) to be available. If you are a visual thinker, then draw a 2D matrix on a piece of paper. Label the horizontal axis as the weights (w) and the vertical axis as the items available (i). The top-left corner denotes (0,0) and the bottom-right corner denotes (W, I ) then arbitrarily pick a cell in the matrix and label it (w, i). Next, draw 2 arrows depicting the direction of where (w wi , i 1) and (w, i 1) would lie. This allows you to visually see the dependencies! So what you want is an ordering so the dependencies will always be fullled by the time you reach that subproblem. Here we have two options, we can calculate in the direction of increasing w (outer loop) and then increasing i (inner loop) or with the direction of increasing i (outer loop) and increasing w (inner loop). An example of the rst order would be (0,0),(0,1), (0,2),(1,0),(1,1),(2,1) etc. An example of the second order would be (0,0),(1,0),(2,0),(3,0),(0,1),(1,1),(2,1),(3,1),(0,2) etc. These assume W = 3 and I (number of items)= 2. Once we decide a valid ordering, implementation is usually straightforward. 16

Figure 2: Bottom-up Dynamic Programming Order Dependency Diagram Have nested for-loops that loop through the order and use an array to cache results much like what we did with memoization. This can be seen in the Code Listing 7. How do we back-track our results from this? You could do what we did last time with the rst variant. However, there exists a more elegant solution where we simply just use our 2-D cache DP/array. How would do this work? We start o at dp(W, I ) now we check if dp(W, I ) is equal to dp(W, I 1). If it is, then we implicitly deduce that item I is not in the best item conguration. If dp(W, I ) is not equal to dp(W, I 1) then we can deduce that item I is in the best item conguration, so we print that item out (or store it) and we backtrack/recurse to dp(W wi , I 1) where wi = weight of item i. A simple implementation of this backtracking algorithm can be seen in Listing 8 and 9. There are many other dierent variations of the knapsack problem one variant allows you to split objects into fractions. This variant is sometimes called fractional knapsack here a greedy strategy suces, convince yourself either informally or formally with a proof. There are also more complicated knapsack problems involving multiple criteria and constraints usually called multidimensional knapsack problems. However these are beyond the scope of this tutorial and usually employ dierent algorithmic techniques to solve.

17

Listing 7: 0-1 Knapsack Bottom-up Dynamic Programming

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 #include < i o s t r e a m > #include < v e c t o r > #include < u t i l i t y > #define MAX WEIGHT 1000 #define MAX ITEMS 50 using namespace s t d ; i n t dp [MAX WEIGHT+ 1 ] [MAX ITEMS+ 1 ] ; i n t main ( ) { i n t weight , v a l u e ; i n t bagWeight ; c i n >> bagWeight ; i f ( bagWeight > MAX WEIGHT) return 1 ; v e c t o r <p a i r <int , int > > data ; while ( c i n >> w e i g h t >> v a l u e ) { data . push back ( m a k e p a i r ( weight , v a l u e ) ) ; } i f ( data . s i z e ( ) > MAX ITEMS) return 1 ; // s t a r t bottom up dynamic programming s o l u t i o n f o r ( i n t w = 1 ; w <= bagWeight ; w++) { f o r ( i n t i = 1 ; i <= data . s i z e ( ) ; i ++) { dp [ w ] [ i ] = dp [ w ] [ i 1 ] ; i f ( data [ i 1 ] . f i r s t <= w) dp [ w ] [ i ] = max( dp [ w ] [ i ] , dp [ wdata [ i 1 ] . f i r s t ] [ i 1]+ data [ i 1 ] . s e c o n d ) ; } } // p r i n t o u t s o l u t i o n c o u t << " Optimal value : " << dp [ bagWeight ] [ data . s i z e ( ) ] << "\n" ; return 0 ; }

4
4.1

Edit Distance
Modelling and Implementation

You have now experienced most of the aspects involving implementing and modelling a dynamic programming algorithm. We have seen the two dierent approaches of implementation, namely top-down dynamic programming (memoization) and bottom-up dynamic programming. We have also seen how recursion

18

Listing 8: 0-1 Knapsack Bottom-up Dynamic Programming with Backtracking

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 #include < i o s t r e a m > #include < v e c t o r > #include < u t i l i t y > #define MAX WEIGHT 1000 #define MAX ITEMS 50 using namespace s t d ; i n t dp [MAX WEIGHT+ 1 ] [MAX ITEMS+ 1 ] ; v e c t o r <int > itemsUsed ; // used f o r b a c k t r a c k i n g v e c t o r <p a i r <int , int > > data ; void b a c k t r a c k ( i n t w, i n t i ) { i f (w == 0 | | i == 0 ) return ; i f ( dp [ w ] [ i ] == dp [ w ] [ i 1]) b a c k t r a c k (w, i 1); else { b a c k t r a c k (wdata [ i 1 ] . f i r s t , i 1); itemsUsed . push back ( i ) ; } } i n t main ( ) { i n t weight , v a l u e ; i n t bagWeight ; c i n >> bagWeight ; i f ( bagWeight > MAX WEIGHT) return 1 ; while ( c i n >> w e i g h t >> v a l u e ) { data . push back ( m a k e p a i r ( weight , v a l u e ) ) ; c o u t << " Item " << data . s i z e ( ) << " Weight : " << w e i g h t << " Value : " << v a l u e << "\n" ; } i f ( data . s i z e ( ) > MAX ITEMS) return 1 ; // s t a r t bottom up dynamic programming s o l u t i o n f o r ( i n t w = 1 ; w <= bagWeight ; w++) { f o r ( i n t i = 1 ; i <= data . s i z e ( ) ; i ++) { dp [ w ] [ i ] = dp [ w ] [ i 1 ] ; i f ( data [ i 1 ] . f i r s t <= w) dp [ w ] [ i ] = max( dp [ w ] [ i ] , dp [ wdata [ i 1 ] . f i r s t ] [ i 1]+ data [ i 1 ] . s e c o n d ) ; } } // p r i n t o u t s o l u t i o n c o u t << " Optimal value : " << dp [ bagWeight ] [ data . s i z e ( ) ] << "\n" ;

19

Listing 9: 0-1 Knapsack Bottom-up Dynamic Programming with Backtracking (continued)

46 47 48 49 50 51 52 b a c k t r a c k ( bagWeight , data . s i z e ( ) ) ; c o u t << " Items Used :\ n" ; f o r ( i n t i = 0 ; i < itemsUsed . s i z e ( ) ; i ++) { c o u t << " Item " << itemsUsed [ i ] << "\n" ; } return 0 ; }

ts into the modelling and formulation of dynamic programming algorithms and the processes involved in creating one. We have also seen a classical example where dynamic programming is useful (knapsack), the reasons why it is used (Fibonacci example) as well as backtracking useful and meaningful information from a computation. We now go to a more advanced example to reinforce the ideas that have been introduced so far. The next problem we will look at is called the Edit Distance problem or more specically its called Levenshteins distance. Basically its a metric to measure the dierences between two strings. The metric usually consists of how many edits we need to make on a string to turn it into the other string. Edits can be either substituting a letter for something else, inserting a new character anywhere in the string or deleting a character. How do we create a dynamic programming algorithm for this problem? Lets see if we can construct any subproblems. If we can then we should be able to translate it to a dynamic programming with ease because itll give us an insight into the decision state of the problem as well as give hints towards the relationship between them (recurrence relation). First, lets create some examples to get a feel of the problem. Consider the following two strings: N O T H I N G S O M E T H I N G How do we turn one to the other with the minimal amount of moves? A rough idea would suggest somehow discarding counting THING at the end of each of the words. So if we wanted to change NO to SOME how would we do it? We have three possible decisions or choices (inserting, deleting and substituting). We need to try a systematic approach instead of trying to solve it by looking (because computers cant solve by looking yet). Since we want subproblems, what could possible subproblems be? In fact, if we look closely we have already made a subproblem! NO and SOME is a subproblem to NOTHING and 20

SOMETHING. How did we get there? We looked at the ends and decided that since they were equal, we skipped them. This is a sucient and good starting point. Lets try making the state being the last characters of the rst and second string we are looking at. For example at the start we let x = 6 and y = 8 (assuming 0-based index), the last characters of their respective strings. Now we compare the characters current in position x and y . If they are equal, we will do what we did implicitly, skip them. So what happens on x = 1 and y = 3 (O and E)? Well we have multiple choices: 1. We use substitution; this will make O become E. This will incur a cost of 1. Now the subproblems for this decision would be x = 0 and y = 2 because we made them the same. 2. We use deletion; we can delete O and hope to nd a better match from string 1 later on. This will incur a cost of 1. Now the subproblems for this decision would be x = 0 and y = 3 because we ignore the O because we deleted it, however we still need to match it to E because we didnt delete that. 3. We use insertion; we insert E after O, so there will be a new match. The second string will decrease by 1 position because the E was matched by the insertion. However, the rst string stays in the same position because we still need to do something with the O, we merely put it o by doing an insertion. So the new subproblem for this decision would be x = 1 and y = 2. Those were our three choices; in fact, they make perfect subproblems because rstly, its systematic and easily implementable. Second, it considers all possible cases where we can modify the string. Third, subproblems depend only on strictly smaller parts of the string and no overlapping occurs, so the order/dependencies can be implemented correctly. So now we have decided our state to be the two positions relative to the positions we are up to in the two strings. Our recurrence relation can be deduced directly from the three cases: F (x 1, y ) + 1 F (x, y 1) + 1 F (x, y ) = min F (x 1, y 1) + d(x, y ) F (x, y ) = 0 if x < 0 or y < 0 d(x, y ) = 0 1 if a[x] = b[y ] otherwise

Note that d(x, y ) evaultes to a cost of 1 if the characters in the current string positions dont match, otherwise d(x, y ) evaluates to a cost of 0 (because there

21

is no need to substitute since they have matching characters). Once our recurrence relation is dened we can implement it. Here we will demonstrate both top-down and bottom-up implementations its also a good time to practice implementation skills, so its a good idea to try and implement your own code based on the above recurrence before looking at the supplied source code. See Listing 10 for a memoization approach to the Edit Distance problem. See Listing 11 for a bottom-up dynamic programming approach to the Edit Distance problem. Listing 10: Edit Distance using Memoization Approach
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 #include #include #include #include <i o s t r e a m > <v e c t o r > <s t r i n g > <c c t y p e >

#define MAX STRING LEN 5000 #define INF INT MAX using namespace s t d ; i n t memo [ MAX STRING LEN+ 1 ] [MAX STRING LEN+ 1 ] ; string s , t ; int d( int x , int y ) { i f ( s [ x 1] == t [ y 1]) return 0 ; return 1 ; } int func ( int x , int y ) { i f ( x == 0 | | y == 0 ) return max( x , y ) ; i n t& r e s = memo [ x ] [ y ] ; i f ( r e s != 1) return r e s ; r e s = INF ; r e s = min ( r e s , f u n c ( x 1,y 1)+d ( x , y ) ) ; r e s = min ( r e s , f u n c ( x 1,y ) + 1 ) ; r e s = min ( r e s , f u n c ( x , y 1)+1); return r e s ; } i n t main ( ) { while ( c i n >> s >> t ) { memset (memo, 1 , s i z e o f (memo ) ) ; c o u t << " Edit distance is : " << f u n c ( s . s i z e ( ) , t . s i z e ( ) ) << "\n" ; } return 0 ; }

22

Listing 11: Edit Distance using Bottom-up Dynamic Programming

1 #include < i o s t r e a m > 2 #include < s t r i n g > 3 4 #define MAX STRING LEN 5000 5 6 using namespace s t d ; 7 8 i n t dp [ MAX STRING LEN+ 1 ] [MAX STRING LEN+ 1 ] ; 9 string s , t ; 10 11 i n t d ( i n t x , i n t y ) { 12 i f ( s [ x 1] == t [ y 1]) return 0 ; 13 return 1 ; 14 } 15 16 i n t main ( ) { 17 while ( c i n >> s >> t ) { 18 memset ( dp , 0 , s i z e o f ( dp ) ) ; 19 f o r ( i n t i = 0 ; i <= s . s i z e ( ) ; i ++) dp [ i ] [ 0 ] = i ; 20 f o r ( i n t j = 0 ; j <= t . s i z e ( ) ; j ++) dp [ 0 ] [ j ] = j ; 21 f o r ( i n t i = 1 ; i <= s . s i z e ( ) ; i ++) { 22 f o r ( i n t j = 1 ; j <= t . s i z e ( ) ; j ++) { 23 dp [ i ] [ j ] = min ( dp [ i 1 ] [ j ]+1 , min ( dp [ i ] [ j 1]+1 , 24 dp [ i 1 ] [ j 1]+d ( i , j ) ) ) ; 25 } 26 } 27 c o u t << " Edit Distance is : " << dp [ s . s i z e ( ) ] [ t . s i z e ( ) ] 28 << "\n" ; 29 } 30 return 0 ; 31 }

4.2

Performance Dierences

We will now discuss the performance dierences between the two approaches should you opt for one over the other? Are there any rough indicators to suggest when one should be used over the other? We will now to a mini-benchmark on the two approaches. After writing a random string generator that generates two strings based on a specied length we feed a collection of such strings into our program (i.e. by piping). After doing a small test of roughly 20 random strings of possible size up to 1000 length, we have the following numbers: Memoization Approach: 1.683s 23

Dynamic Programming Approach: 1.094s Most of the execution time was actually due to the allocation of the large caching/dynamic programming array. If we modied our source code to allocate 1001x1001 sized arrays rather than 5001x5001 arrays the execution times gets cut to: Memoization Approach: 0.941s Dynamic Programming Approach: 0.488s So certainly it seems the bottom-up approach runs more eciently than the memoization approach in this test. For further tests, we bumped up the maximum string length to 5000. We roughly collected around 30 of such strings and the same strings on our two dierent programs. Here, the algorithm runs much slower than it did previously: Memoization Approach: 44.563s Dynamic Programming Approach: 14.295s Here we can see that the bottom-up approach becomes more ecient as the input size grows. Why? This is due to the overhead for recursive calls for the memoization approach, it really magnies as you have larger inputs because it takes a longer time before the caching mechanism kicks in for repeated calls. In fact, it becomes rather dangerous to use memoization due to the risk of stack overow from a deep recursive tree. So you should denitely opt to use bottomup dynamic programming if you can, but it generally isnt as large of an issue as it seems to be. Chances are, if the time required grows exponentially then either of the approaches would be too slow to work in practice for even modest sized inputs. For strict time limits (such as algorithmic competitions) there may be cases where the bottom-up approach will pass within the time limit whilst the memoization approach wont. A general indicator to use a bottom-up approach is when the recursive tree becomes too deep. This can be due to the fact that further subproblems only reduce the parameters by a small amount (in this case by 1, since f (W, I ) calls f (W 1, I 1), f (W 1, I ), f (W, I 1)). So the recursive tree doesnt back out for a long while you can think of this as a long depth rst search tree (DFS has the same problem as memoization), hence it results in a deep recursive tree. So when should you use memoization over the bottom-up approach? These usually occur when only a minority of the subproblems of a problem is important. Since the bottom-up approach calculates all subproblems up to the parameters of a problem whether it is needed or not, this can result in many unnecessary 24

calculations. Here, memoization shines because it only calls the subproblems that it needs from the recurrence relation. Calculating binomial coecients n k is a good example. If we tried to calculate 1000 , then the classical bottom-up 4 approach would need to calculate all of Pascals triangle up to the row 1000 (including all unnecessary columns). In this case, the number of calculations required for the bottom-up approach would always be: n(n 1)/2 (arithmetic series), whereas the number of calculations required by the memoization would be much less than this (the number of calculations required are actually variable, it would require more as the k becomes closer to n/2 but even still, it would be less than the bottom-up approach).

Overview

We have looked at a range of issues involving the modelling, implementation and properties of dynamic programming algorithms through the use of classical examples. This tutorial should equip you with a better sense on how dynamic programming algorithms works as well as a general idea on how to solve one. As with most things in life, only practice can help you fully understand this powerful and general algorithmic tool. There will be a further tutorial based on non-classical algorithms that tackles more challenging problems than the ones seen in this tutorial so far, it will also introduce some neat implementation tricks that you will nd useful.

Problems

Practice what was discussed in this tutorial with a set of elementary and classical dynamic programming problems! Note: These are roughly in the order of diculty. These problems have no strict input structure, choose one that you like and stick with it.

6.1

Problem 1: Longest Increasing Subsequence

Diculty: Easy John likes to play with numbered blocks everyday, what he is particularly interested in is nding increasing subsequences of the blocks. In particular, he would like to know the longest length of such increasing subsequences preferably with an example to re-create such a subsequence. A subsequence isnt necessarily contiguous, for example, today John found that his numbered blocks were in the following order: 3, 2, 9, 5, 6, 8. An example of an increasing subsequence is 5, 6, 8 or 2, 5 (note that the 2 and 25

5 are not contiguous in the sequence). In this example, John helps you and points out that the length of the longest increasing subsequence is actually of length 4: 2, 5, 6, 8 or 3, 5, 6, 8. Unfortunately as smart as John is, his having trouble working out the longest increasing subsequence for a large sequence (up to a sequence length of 10,000). Help him out by writing a program that prints out the length of the longest increasing subsequence, you must also print out a subsequence that corresponds to this length (by using backtracking or other means). Possible le/input structure: Line 1: Length of Sequence (n) Line 2: Followed by n integers Example of such a structure (from Johns example): 6 3 2 9 5 6 8 Example output: Length: 4 Subsequence: 2 5 6 8 Hint: The intended complexity is O(n2 ) time. It can actually be solved in O(n log(n)) time but this is more complex than the idea required here. The time complexity gives a hint on the structure of the program (think of the for loops!). Bigger Hint: A possible state for the problem is F (S ) = the longest increasing subsequence with the last element to be index S . Even Bigger Hint: Recurrence relation is: F (S ) = max{F (S ), F (n) + 1}, if A[n] < A[S ] where n < S

6.2

Problem 2: Binomial Coecients

Diculty: Easy John likes to count things, unfortunately for him he doesnt know how to count the number of ways to choose k items from a set of n items. Write a program that does this for him given input n and k . 26

Background: This is simply a basic mathematical problem relating to binomial coecients. Dene n k as the number of ways to choose k items from a set of n items. Then we can dene a recurrence relation based on the combinatorial argument: you can either choose item k in which there will be n 1 items remaining with k 1 more to choose, or you can not choose item k in which there will be n 1 items remaining with k more to choose. Now we can produce a recurrence relation: n k n1 n1 + k k1

Hint: Part of the recurrence relation is dened for you, the only thing left is to determine the base cases and simply implement it! For our purposes, n 0 will always equal 1, because there is one way to choose nothing: dont choose anything. Bigger Hint: Base cases to be determined: Even Bigger Hint:
n n n n

n 0

. Optional base case:

n 1

= 1,

n 1

= n,

n 0

= 1.

6.3

Problem 3: Longest Common Subsequence

Diculty: Easy-Medium John has recently became interested in subsequences, as evidently in Problem 1. Since you have solved that problem for him, John now gives you a harder problem that he needs help with. He has two character strings, and wishes to know the length of the longest common subsequence of the two strings. Like in Problem 1, a subsequence isnt necessarily contiguous. John gives you an instance of the problem he has solved, the two strings are DYNAMIC and PROGRAMMING. He states that the longest common subsequence is 3: A, M, I. Not convinced, you ask him to give you another example. He gives you another: CLASSICAL and CAPITAL, the longest being 5: C, A, I, A, L. Now, its your turn to write a program that solves the problem, given two strings nd the longest common subsequence. Optionally, to impress John even more also print out an example of such a subsequence (i.e. AMI and CAIAL in the two examples). Hint: This is similar to the Edit Distance problem, start from the back and work towards the front. Bigger Hint: This is a 2-D Dynamic Programming problem (i.e. 2D caching/DP array). Even Bigger Hint: The state is: F (X, Y ) = the length of the longest common 27

subsequence using string 1 from index 0 to X and string 2 from index 0 to Y. Think of how the Edit Distance problem works, when two characters match we added 0, but in this case we want it to add 1 to the length (since we are looking for common elements). If two characters dont match, we recursively check subproblems by deletion/inserting, here we want the maximum of these subproblems. Try and formulate the recurrence relation now.

6.4

Problem 4: Handshaking Problem

Diculty: Medium John has recently joined a secret cult of, handshakers! In a cult meeting, each member is required to shake hands with exactly one other cult member in a circular table. All handshakes happen at the same time because it would take too long to do so otherwise. John has nished the meeting and has returned home safely, now he wonders how many ways were there of shaking hands if no handshake can cross another. There were no clones, so each person is dierent and distinguishable, so you can rotate a shaking conguration to yield a dierent shake. Given the number of cult members there were (n), write a program that returns the number of handshake congurations that could of taken place in the meeting. You may assume there are an even number of cult members (otherwise there would be a loner). Some examples: For 2 Cult Members (John and the cult leader), there is only 1 shake conguration John just shakes the Cult Leaders hand. For 4 Cult Members, there are three possible shaking congurations, only 2 of them are valid. Hence there are two ways to shake. The diagram below shows this, only the rst two are valid because the third one requires handshakes to cross which is invalid. The next day, John asked the cult leader for hints (the Cult Leader is really good with numbers). The cult leader replied For 50 cult members, there are 4861946401452 possible shaking congurations!. Hint: Draw diagrams and try and visualise the problem before doing anything else! Bigger Hint: This is actually a case of Catalan Numbers, however you dont need to know this to solve it. If your stuck, you can just read about Catalan numbers and see how it relates here. As a consequence, the Catalan numbers recurrence is the same as what we need for this problem. 28

Figure 3: 4 Cult Member Case

End of Tutorial
If you have any feedbacks, suggestions or corrections please feel free to contact me on my e-mail address specied in the front of this tutorial.

29