Recurrence Relations Recursive Algorithm Recursive algorithm invokes (makes reference to) itself repeatedly until a certain condition

matches Examples: Computing factorial function Tower of Hanoi puzzle Digits in binary representation Non-recursive algorithm is executed only once to solve the problem. Steps in mathematical analysis of recursive algorithms: Decide on parameter n indicating input size. Identify algorithms basic operation Set up a recurrence relation and initial condition(s) for T(n)-the number of times the basic operation will be executed for an input of size n o Alternatively count recursive calls. Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution Recurrence relations result naturally from the analysis of recursive algorithms, specifying the algorithm's run time; solving recurrence relations yields a closed-end formula for calculation of run time. Why are recurrences good things? 1. Many natural functions are easily expressed as recurrences:

2. It is often easy to find a recurrence as the solution of a counting problem. Solving the recurrence can be done for many special cases as we will see, although it is somewhat of an art.

For example, the recursive n! algorithm of: int factorial (int n) { if (n == 1) return 1; else return factorial (n-1) * n; } The running time, T(n), can be defined as:

T(1) = 1 T(n) = T(n-1) + 4 for all n>1 Eg. T(n) = 2 * T(n=2)+n is an example of a recurrence relation A Recurrence Relation is any equation for a function T, where T appears on both the left and right sides of the equation. We always want to solve these recurrence relations by getting an equation for T, where T appears on just the left side of the equation. Solving Recurrence Relations We study here, three methods of solving recurrence relation Forward Substitution Backward Substitution Homogeneous Equation method

Forward Substitution Method Consider a recurrence relation such as: t(n) = 2t(n/2) + n t(1) = 0 This is not a "closed-form". A closed-form is a function like t(n) = nlog n (which is the answer for the recurrence relation above). The recurrence relation is an intermediate step in the process of determining the big-O running time of an algorithm. After determining the recurrence relation we need to find a closed-form. The simplest way to solve easy recurrence relations it to guess a solution and prove it by induction. How do you guess a solution? There are two ways. One is to plug in small values of n and deduce a pattern. This is the one we will discuss here.

Examples 1. t(n) = t(n-1) + 1 (eqn 1) t(1) = 1 (eqn 2) Plugging in: t(2) = t(2-1) + 1 (by eqn 1) = t(1) + 1 = 1 + 1 = 2 (by eqn 2) t(3) = t(2) + 1 = 2 + 1 = 3 Now we guess that t(n) = n, which is correct

After you guess the closed form for a recurrence relation you need to prove it correct by induction. The proof will work out if and only if your guess is correct (assuming you are doing inductive proofs correctly). If it doesn't work you need to change your guess.

Proof Claim : For the recurrence relation t(n)= n Proof (by induction) Basis: We claim that t(1) = 1, which is the value given by the recurrence relation Inductive Hypothesis: Assume t(n-1) = (n-1) for some n Inductive Step: We must show that t(n) = n t(n)= t(n-1)+1 =( n-1) + 1 by inductive hypothesis =n Therefore the claim is true

2.

t(n) = t(n-1) + n t(1) = 1

Plugging in: t(2) = t(1) + 2 = 1+2=3 t(3) = t(2) + 3 =3+3=6 t(4) = t(3) + 4 = 6 + 4 = 10 This one is actually tricky unless you notice that the sum is 1 + 2 + ... + n =n(n+1)/2. Proof Claim: For this recurrence relation, t(n) = n(n+1)/2 Proof (by induction): Basis: We claim that t(1) = 1(2)/2 = 1, which is the value given by the recurrence relation so the claim is true for n = 1. Inductive Hypothesis: Assume t(n-1) = (n-1)*n/2 for some n. Inductive Step: We must show that t(n) = n(n+1)/2. t(n) = t(n-1) + n = (n-1)*n/2 + n by I.H. = (n*n - n + 2n)/2 = (n2 + n)/2 = n(n+1)/2 Therefore the claim is true. 3. t(n) = t(n/2) + n t(1) = 1

When dealing with n/2 you may consider only values that are powers of 2 in this class. Plugging in: t(2) = t(1) + 2 = 1 + 2 = 3 t(4) = t(2) + 4 = 3 + 4 = 7 t(8) = t(4) + 8 = 7 + 8 = 15 t(16) =t(8) + 16 = 15 + 16 = 31 ------^ Notice that in the 3rd column we find that t(n) = (n-1) + n = 2n-1. This is correct. Give Proof yourself Practice Problems: Guess solutions for the following recurrence relations. 1. t(n) = t(n-1) + n; t(1) = 1 2. t(n) = 2t(n/2) + 1; t(1) = 0 3. t(n) = t(n/2) + n; t(1) = 1 For each of the recurrence relations below, find the solution 1. t(n) = t(n-1) + 2; t(1) = 1 2. t(n) = t(n/2) + n; t(1) = 1; 3. t(n) = 2*t(n/2) + 3n; t(2) = 1

Backward Substitution Method T(n) = 1 , if n = 1,

2T(n 1) + 1, if n > 0 Solution: STEP 1: Apply enough substitutions to find a pattern i = 1 T(n) = 2 T(n 1) + 1 i = 2 T(n) = 2[2T(n 2) + 1] + 1 = 22T(n 2) + 2+ 1 i = 3 T(n) = 22 [2T(n 3) + 1] + 2 + 1 = 23 T(n 3) + 22 + 2 + 1 i = 4 T(n) = 23 [2T(n 4) + 1] + 22 + 2 + 1 = 24T(n 4) + 23 + 22 + 2 + 1 .. STEP 2: Guess what the recurrence is after i substitutions: T(n) = 2iT(n i) + (2i + 2i-1 + 2i-2 + . + 22 + 2 + 1)

Sum of (2i + 2i-1 + 2i-2 + . + 22 + 2 + 1) = (2i 1)/ 2 -1

STEP 3: Solve for i when the recurrence reaches a base case: For this problem, when n i = 1 (since T(1) is the base case) or i = n 1. STEP 4: Substitute the value for i back into the equation. T(n) = 2n1T(1) + 2n1 1 = 2n1 + 2n1 1 T(n) Example 2 Recurrence: T(n) = 7, T(n 1) + c, Solution STEP 1: Apply enough substitutions to find a pattern i = 1 T(n) = T(n 1) + c i = 2 T(n) = T(n 2) + 2c i = 3 T(n) = T(n 3) + 3c i = 4 T(n) = T(n 4) + 4c STEP 2: Guess the recurrence after i substitutions: T(n) = T(n i) + ic STEP 3: We terminate when the recurrence reaches a base case - for this problem, when n i = 3 or i = n 3. STEP 4: Substitute back into the recurrence guessed in STEP 2: T(n) = T(3) + (n 3)c T(n) = 7 + (n 3)c Hence solved. Given a Recurrence, the Open form of the equation. T(1) = 1 T(n) = 2T(n/2) + n Backward Substitution Find T(n) in terms of i by using Backward Substitution and looking for a pattern. T(n) = 2T(n/2) + n if n = 3, if n > 3 = 2n 1

= 2[2T(n/22) + n/2] + n = 22T(n/22) + n + n = 22T(n/22) + 2n After ith substitution T(n) = 2iT(n/2i) + in When reaching the base case, T(1) will need to equal the recursive call. So set 1 equal to the reduction of n in terms of i and solve for i. Base: T(1) = T(n/2i) 1 = n/2i 2i = n log2 n = i T(1) = T(n/2log2n) = T(n/n) = T(1) Use T(n) in terms of i from and substitute i . Reduce algebraically until you get a Closed form for T(n). Closed Form: T(n) = 2iT(n/2i) + in = 2log2 nT(n/2log2 n) + (log2 n)n = nT(n/n) + nlog2 n = nT(1) + nlog2 n T(n) = n + nlog2 n Note: logb x = y by = x

Solving Linear homogeneous recurrence of degree k tn = b1tn-1 + b2tn-2 +...+bktn-k b1,b2,...,bk are real numbers bk 0.

Linear : sum of constant multiples of the previous terms of the sequence. Homogeneous : no terms that are not multiples of tj s. Degree k : tn is expressed in terms of previous k terms of sequence. Sequence uniquely determined by recurrence relation and k initial conditions t0 = C0, t1= C1, ..., tk-1 = Ck-1 Why important? often occur in determination of run-time of algorithms. can be systematically solved, without guesswork Solving linear homogeneous recurrences The recurrence can be rewritten as tn - b1tn-1 - b2tn-2 - ...- bktn-k = 0 or a0tn + a1tn-1 + a2tn-2 +...+aktn-k = 0 where a0 =1,a1 = -b1,a2 = -b2,...,ak = -bk

Note that any linear combination of solutions is itself a solution. Look for solutions of form tn = xn where x is a constant. Substituting: a0 xn +a1xn-1 +...+akxn-k = 0 i.e. xn-k (a0xk + a1xk-1 +...+ak ) = 0 Satisfied if xn-k=0 - no interest to us. Otherwise, equation satisfied if and only p(x) = a0xk+a1xk-1+...+ ak = 0 characteristic equation p(x) is characteristic polynomial p(x) = where ri may be complex numbers.

These ri are only solutions of the equation p(x)=0.

Therefore tn =

satisfies the recurrence for any choice of constants c1,c2,...,ck . All the ri distinct only solutions of this form.

Constants can be determined from k initial conditions by solving system of k linear equations in k unknowns. Example Fibonacci numbers: f0 =0 f1 =1 fn = fn-1 fn-2 for n >1 Rewriting: fn - fn-1 - fn2 =0

Characteristic polynomial: x2 - x - 1 Characteristic equation: x2 - x 1=0 Roots: r1 =(15 )/2 , r2 =(15 )/2 General solution: fn = c1r1n c2r2n n=0 : f0 = c1 c2 0 n=1 : f1 = c1r1 c2r2 1 Solving: Thus: fn c1 = 1/5, c2=1/5

1/5 ( (1+5) n /2) (1-5) n /2 )

Another example t0 = 0 t1 = 5 tn = 3tn1 + 4tn2 if n >1 Rewriting: tn 3tn1 4tn2 = 0 Characteristic equation: x2 3x 4 = (x +1)(x 4) Roots: r1 = 1, r2 = 4.

General solution: tn = c1(1)n + c24n Initial conditions: c1 + c2 = 0 when n=0 c1 + 4c2 = 5 when n=1 Solving: c1 = 1, c2 =1. Therefore: tn = 4n (1)n