COT 5405 Analysis of Algorithms, Spring 2010.

Homework 3

Due Friday, April 2 2010, 12:00pm.

Notes
• You may hand the homework to TAs during office hours. You may also hand the homework to Prof.
Ranka in class. No late submissions.
• To force you to write succinctly, we have enforced page limits. These are noted in front of each question.

• Answer each question on a fresh page.

• If the problem necessitates writing an algorithm, you must first informally describe the algorithm, in
brief, in a paragraph. You can choose to follow this up with pseudocode that formally describes the
algorithm. We will peruse your pseudocode only if your English description is not clear.
• Write your name on the top right hand corner of your homework. Be sure to write your last name as
the last word in your name.
• If you are designing an algorithm, you must write a formal proof of correctness.

• Please write legibly.

1
1. [1 page][15pts] You are given n matrices M1 , M2 , ..., Mn of sizes s0 × s1 , s1 × s2 , ..., sn−1 × sn re-
spectively. Suppose you want to compute their product M1 × M2 × ... × Mn . Matrix multipli-
cation is not commutative (i.e. M1 × M2 6= M2 × M1 ), but we do know it is associative (i.e.
M1 × (M2 × M3 ) = (M1 × M2 ) × M3 ). It turns out that the order in which you compute the product
is important because for some such orders, the number of operations (i.e. number of multiplications)
required is less than that for other orders. For example, if you have four matrices M1 , M2 , M3 , M4 of
sizes 40 × 25, 25 × 1, 1 × 11, 11 × 101 respectively, then (M1 × M2 ) × (M3 × M4 ) is the best ordering
as it takes only (40 × 25 × 1) + (1 × 11 × 101) + (40 × 1 × 101) = 6151 multiplications. The ordering
(M1 × ((M2 × M3 ) × M4 )) is the worst. Remember that multiplying a matrix of size a × b with another
of size b × c takes O(abc) multiplications using the naive method (we aren’t going to consider any faster
algorithms like Strassen’s method).
It turns out that for this problem, the greedy approach of choosing the available matrix pair whose
product is the cheapest to compute, is suboptimal. Give an example to illustrate this.
Next, prepare an efficient dynamic programming strategy to find the best possible ordering to compute
the product of the n given matrices. Write a recurrence relation, explain the algorithm and then write
a non-recursive pseudocode. Analyze the complexity of your algorithm. (It turns out that there is an
O(n3 ) algorithm for this problem).

Solution:

Counterexample for greedy strategy: In the example given in the problem, the pair M23 = M2 × M3
is the cheapest, needing only 25 × 11 multiplications. This is followed by M123 = M1 × M23 takes
40 × 25 × 11 multiplications and lastly M123 × M4 takes another 40 × 11 × 101 multiplications. The
total is 55715 (suboptimal).
Let us denoted by R(i, j) the optimal cost of computing Ai ×...×Aj where 1 ≤ i ≤ j ≤ n. Clearly for all
i, R(i, i) = 0. To compute R(i, j) we need to find an index k (i ≤ k ≤ j) to split the multiplication chain.
Then we have R(i, j) = R(i, k) + R(k, j) + si−1 × sk × sj . The values R(i, k) and R(k, j) will be com-
puted recursively. Naturally, we want the index k such that R(i, k) + R(k, j) + si−1 × sk × sj is the least
possible amongst all indices between i and j, i.e. R(i, j) = mink;i≤k<j R(i, k) + R(k, j) + si−1 × sk × sj .
We thus write a pseudocode as follows:
MATRIX-MULTIPLY-CHAIN (n, M1 , M2 , ..., Mn , s0 , s1 , ..., sn )
FOR i = 1 TO n, R(i, i) = 0 END
FOR q = 1 TO n − 1
FOR i = 1 TO n − s
j =i+q
R(i, j) = mink;i≤k<j R(i, k) + R(k, j) + si−1 × sk × sj
END (close first for loop)
END (close second for loop)
This is an O(n3 ) algorithm as there are two outer for-loops each taking O(n) time.

2
2. [1 page][15pts] Given a set S of n integers, write a dynamic programming algorithm to check whether
there exists a subset of S whose sum equals W where W is an integer. Your algorithm should run
in time O(nW ). Write the recurrence, explain the algorithm and write a non-recursive pseudocode.
Explain whether this can be called a polynomial time algorithm. Extend the algorithm
P to determine
P
whether there exist two subsets S1 and S2 such that S1 ∪ S2 = S, S1 ∩ S2 = φ and x∈S1 x = x∈S2 x.

Solution:

The first part of the problem is called the subset-sum problem.

The original set is called S and let sk be the k th element of S. We know that |S| = n. To solve this
problem, we maintain a Boolean table C. Here C(i, w) = 1 if there is a subset of the first i elements
of S which sums up to w, else C(i, w) = 0.
Observe that if sn > W , then sn cannot be part of the subset we are looking for. In such a case, we
can conclude that C(n, W ) = C(n − 1, W ). If sn ≤ W , then we have two choices: (1) We can include
sn in our desired subset and then check the subproblem restricted to the first n − 1 elements for a
weight W − sn . In this case, we have C(n, W ) = C(n − 1, W − sn ). OR (2) Do not include sn in the
desired subset and refer to the subproblem restricted to the first n − 1 elements for a weight W . Either
of these choices is fine, as long as at least one choice gets us a 1.
This notion is summarized by the following recurrence relation:
If sn > W , then C(n, W ) = C(n − 1, W ), else C(n, W ) = C(n − 1, W − sn ) OR C(n − 1, W ))
The pseudocode is as follows:

SUBSET-SUM (n, W )
FOR w = 1 TO W , C(0, w) = 0, END
FOR i = 1 TO n
FOR w = 0 TO W
IF si > w, C(i, w) = C(i − 1, w);
ELSE
C(i, w) = C(i − 1, w − si ) OR C(i − 1, w))
ENDIF (close IF)
END (close second FOR)
END (close first FOR)

The complexity of this algorithm is clearly O(nW ). No, it is not a polynomial time algorithm because
the complexity is dependent upon the value of the integer W . Time complexity is always measured in
terms of the size of the input. To store the value W , it takes at the most dlog2 W e bits. The value W
is exponential in dlog2 W e and hence this is an exponential time algorithm.

The extension can be done by calling SUBSET-SUM (n, P ) where P is the half the sum of all el-
ements in S. If P is not an integer we already know that there cannot exist subsets S1 and S2 which
obey the given constraint.

3
3. [1 page][15pts] A new railway route is being set up from city A to city B. There are totally n proposed
sites along that route, on some of which a railway station is to be built. For each site i (1 ≤ i ≤ n),
you know its distance di from city A. Assume that d1 ≤ d2 ≤ ... ≤ dn−1 ≤ dn . The government is
willing to provide fund of p1 , p2 , ..., pn for the constructing a station at these n sites respectively (the
p values are not necessarily in increasing order, unlike the d values). Owing to some regulations laid
down by the government, no two railway stations may be located at a distance of less than or equal to
15 miles from one another. Your job is to come up with an efficient dynamic programming algorithm to
compute the number of railway stations and their locations that maximize the total amount of money
that the government will be willing to pay, subject to the aforementioned constraint. Write the rele-
vant recurrence relation, a non-recursive pseudocode and analyze the time complexity of the algorithm.

Solution:

Let C(j) be the maximum amount that can be possibly paid by the government for constructing
railway stations from sites 1 to j (for the optimal strategy). We know that for any site j, either a
railway station will be constructed at that site or it won’t. In the former case, the total donation earned
will be C(j) = pj + C(l(j)) where l(j) is the first site to the left of j which is located at a distance
more than 15 miles away from site j (of course l(j) < j). In the latter case, C(j) = C(j − 1). Thus
the maximum total amount that can be collected for sites 1 to j is C(j) = max(C(j − 1), pj + C(l(j)).
Carrying out this recurrence requires that you know l(j) for all j from 1 to n. To compute the l values,
start from dn . Find the rightmost index that obeys the constraint and call it as x. Thus l(n) = x. For
dn−1 , check whether dn−1 − dx ≥ 15. If so, l(n − 1) = x, else decrement x till the condition is satisfied
and set l(n − 1) = x then. Repeat this for other indices. For those indices of the distance array for
which no other index satisfies the constraint, set l(i) = 0 (it means that a railway station cannot be
constructed at site i). Computing the l values clearly takes O(n) time, as for any pair of indices j1
and j2 , we have j1 ≤ j2 ↔ l(j1 ) ≤ l(j2 ).
The pseudocode can be given as follows:

BUILD-RAILWAY-STATIONS (n, d1 , ..., dn , p1 , ..., pn )

FOR i = 1 TO n
C(i) = 0;
S(i) = 0;
END

FOR i = 1 TO n
C(i) = max(C(i − 1), pi + C(l(i))
IF C(i − 1) ≤ pi + C(l(i)), S(i) = 1 ELSE S(i) = 0; END
END

/// ∗ ∗ Comment: The part above gives you the maximum profit in array C that you can get by fol-
lowing the optimal policy. The following part keeps track of a binary array S which tells you whether
or not a station was built at each location. ∗ ∗ //
i=n
WHILE (i > 0)
BEGIN
IF pi + C(l(i)) > C(i − 1), S(i) = 1 ELSE S(i) = 0 END
i = l(i)
END

4
The complexity of this algorithm is clearly O(n). In the above pseudocode S is a binary array. S(i) = 1
if a station will be constructed at site i as per the optimal strategy, else S(i) = 0.

5
4. [1 page][15pts] Pearls have been prized for their beauty and rarity for more than four thousand years.
Natural pearls are collected in the coastal regions of Asia, Africa and America. Since pearl collection
is dangerous, researchers are asked to build a robot that can do this task. A task for the robot consists
of n potential pearl sites all located along a single coastal line. Robot starts at site 1, but can finish at
any site. To make things easier we assume that all the sites are on a straight line, one after the other.
The time taken to travel from site i to site i+1 is ti,i+1 hours, 1 ≤ i ≤ n − 1. Assume that robots
battery lasts for m hours. Also, the expected number of pearls per hour is pi at site i, 1 ≤ i ≤ n. It
was noticed that each hour of pearl collection decreases the expected number of pearls to be found in
the next one hour, by a constant amount. This amount varies from site to site and is given as di , for
site i, 1 ≤ i ≤ n. Design a dynamic programming algorithm to maximize the total pearls found. Give
the pseudo-code and run time complexity.

Solution:

Since the rate of decrease at each site is different, we need to check all possibilities of spending
available battery time. At each site, we calculate the number of pearls found after each time unit.
T [i, j], 1 ≤ i ≤ n, 1 ≤ j ≤ m represents the total pearls found when the robot is at site i and the
battery time used is j (n is number of sites and m is number of hours available). The base values,
T [1, j] can be calculated in O(m) time. t[i, j] represents the number of pearls found if robot spends j
units of time at site i. It can be calculated in O(nm) time. Let Di be the total time taken to reach
site i. Note that D0 = 0.
T [i, j] = maxDi ≤k≤j {T [i − 1, j − k] + t[i, k − Di ]}
i varies from 1 to n, j varies from 1 to m and k also varies from 1 to m (since it depends on j). So the
runtime complexity for this algorithm is O(nm2 ).

6
5. [1 page][15pts] Your favorite sawmill charges by length to cut each board of lumber. For example, to
make one cut anywhere on an 8 ft. board of lumber costs $8. The cost of cutting a single board of
wood into smaller boards will depend on the order of the cuts. As input, you are given a board of
length n marked with k locations to cut. Design a dynamic programming algorithm that, given an
input length n of wood and a set of k desired cut points along the wood, will produce a cutting order
with minimal cost in O(k c ) time, for some constant c.

Figure 1: Example

Solution:

For this problem, we let the values of the k cuts be c[1...k]. Let c[0] = 0 and c[k + 1] = n. Then
the board is partitioned into k + 1 regions, where region ri goes from c[i − 1] to c[i] and has length
c[i] − c[i − 1]. The overlapping subproblems part is easy to see. The key to using dynamic programming
is to notice that at any point in a solution, you have a set of boards, each of which is the union of
consecutive subregions.
Let A[i, j] be the cost of optimally cutting the region consisting of ri , ri+1 , ..., rj . Then we can recur-
sively define A[i, j] as follows:
A[i, j] = c[j] − c[i − 1] + mini≤k<j {A[i, k] + A[k + 1, j]}
The first term in this expression is the cost of making a cut in region ri , ri+1 , ..., rj , and the second is
the cost of the recursive solution. We use the following algorithm, which fills in the table, one diagonal
at a time. We use the notational conventions mentioned above:
CU T (n, k, c[1, ...k])
c[0] ← 0
c[k + 1] ← n
for i = 1 to k + 1
A[i, i] ← 0
for dif f = 1 to k
for i = 1 to k + 1 − dif f
j ← i + dif f
A[i, j] ← (c[j] − c[i − 1]) + mini≤k<j {A[i, k] + A[k + 1, j]} Each of O(k 2 ) cells in the table is defined
by O(k) other cells, so the running time of this algorithm is O(k 3 ).

7
6. [1 page][15pts] Assume that n programs are toPbe stored on two tapes. Let li be the length of tape
needed to store the ith program. Assume that li ≤ L, where L is the length of each tape. A program
can be stored on either of the two tapes. If S1 is P
the set ofPprograms on tape 1, the the worst-case
access time for a program is proportional to max{ i∈S1 li , i6∈S1 li }. An optimal assignment of pro-
grams to tapes minimizes the worst-case access times. Formulate a dynamic programming approach
to determine the worst-case access time of an optimal assignment. Write an algorithm to determine
this time. What is the complexity of your algorithm?

Solution:
P P P
If the summation of the total program is t = i li , the worst case access time max{ i∈S1 li , i6∈S1 li }
will be minimized when,
t X
| − li | (1)
2
i∈S1

can be reduced as possible. If li was a real number, to minimize (1) would be a NP-complete, but li is
a positive integer (because li is the length of a program), which can be solved in O(nt) time complexity
and O(t) space complexity with dynamic programming technique.
For dynamic programming, let’s define a n × (d 2t + 1e) P sized boolean table. For 0 ≤ i < n and
0 ≤ j ≤ d 2t e, when we assigned up to ith program and k∈S1 lk can be j, then T [i, j] is ”TRUE”,
otherwise ”FALSE”.
½
T RU E, if j = 0 (when program l 6∈ S1 ) or j = l0 (when program l ∈ S1 )
T [0, j] =
F ALSE, otherwise
If l is larger than 0 and T [i − 1, j] is determined for all j,T [i, j] will be,

T [i, j] = T [i − 1, j] ∨ (T [i − 1, j − li ] ∧ (j − li ≥ 0))

T [i − 1, j] represents when S1 doesn’t have the program i, and T [i − 1, j − li ] does when S1 do have
the program i.
With above method, after filling out from 0 row to (n-1) row, we can realize the worst access time by
reading the last row. In the last row where the all n programs are assigned to the two tapes, it is just
to find the maximum j form the (n-1)th row such that T [n − 1, j] is ”TRUE”. If you call such j as J,
the worst access time would be (t − J).
The time complexity if O(nt) and the space complexity is O(t).

8
7. [1 page][10pts] Let L be an array of n distinct integers. Give an efficient algorithm to find the length
of a longest increasing subsequence of entries in L. For example, if the entries are 11, 17, 5, 8, 6, 4, 7,
12, 3, a longest increasing subsequence is 5,6,7,12. What is the run time of your algorithm?

Solution:

Define a table with cells c[i, j], for 0 ≤ i, j ≤ n. The cell c[i, j] stores the longest monotonically
increasing subsequence of the prefix Xi =< x1 , x2 , ..., xi > such that each element of the sequence is at
most xj . Fill in the appropriate boundaries of the table and fill in the innards of the table as follows:
f or(i ← 0; i ≤ n; i + +)
f or(j ← 0; j ≤ n; j + +)
if (xi > xj )
c[i, j] ← c[i − 1, j]
else
c[i, j] ← max(c[i − 1, j], c[i − 1, i] + 1)
The idea is if xi > xj , then xi cannot be used in a sequence where every item is at most xj . So we
use the longest sequence up to i − 1. Otherwise, xi is allowed, but it might still be optimal to omit
it. If we don’t use xi , then, as before, we want a sequence on Xi−1 bounded by xj . If we do use xi ,
then we want a sequence on Xi−1 bounded by xi , which we extend by a single element (xi ). The time
complexity is O(n2 ).