Master Theorem
170
CHAPTER 5. RECURSION AND RECURRENCES
5.2
The Master Theorem
Master Theorem
In the last section, we saw three different kinds of behavior for recurrences of the form aT (n/2) + n if n > 1 d if n = 1.
T (n) =
These behaviors depended upon whether a < 2, a = 2, and a > 2. Remember that a was the number of subproblems into which our problem was divided. Dividing by 2 cut our problem size in half each time, and the n term said that after we completed our recursive work, we had n additional units of work to do for a problem of size n. There is no reason that the amount of additional work required by each subproblem needs to be the size of the subproblem. In many applications it will be something else, and so in Theorem 5.1 we consider a more general case. Similarly, the sizes of the subproblems don’t have to be 1/2 the size of the parent problem. We then get the following theorem, our first version of a theorem called the Master Theorem. (Later on we will develop some stronger forms of this theorem.) Theorem 5.1 Let a be an integer greater than or equal 1. Let c be a positive real number and d a nonnegative form aT (n/b) + nc T (n) = d then for n a power of b, 1. if logb a < c, T (n) = Θ(nc ), 2. if logb a = c, T (n) = Θ(nc log n), 3. if logb a > c, T (n) = Θ(nlogb a ). Proof: In this proof, we will set d = 1, so that the bottom level of the tree is equally well computed by the recursive step as by the base case. It is straightforward to extend the proof for the case when d = 1. Let’s think about the recursion tree for this recurrence. There will be logb n levels. At each level, the number of subproblems will be multiplied by a, and so the number of subproblems at level i will be ai . Each subproblem at level i is a problem of size (n/bi ). A subproblem of size n/bi requires (n/bi )c additional work and since there are ai problems on level i, the total number of units of work on level i is ai (n/bi )c = nc ai bci = nc a bc i to 1 and b be a real
CHAPTER 5. RECURSION AND RECURRENCES
5.2
The Master Theorem
Master Theorem
In the last section, we saw three different kinds of behavior for recurrences of the form aT (n/2) + n if n > 1 d if n = 1.
T (n) =
These behaviors depended upon whether a < 2, a = 2, and a > 2. Remember that a was the number of subproblems into which our problem was divided. Dividing by 2 cut our problem size in half each time, and the n term said that after we completed our recursive work, we had n additional units of work to do for a problem of size n. There is no reason that the amount of additional work required by each subproblem needs to be the size of the subproblem. In many applications it will be something else, and so in Theorem 5.1 we consider a more general case. Similarly, the sizes of the subproblems don’t have to be 1/2 the size of the parent problem. We then get the following theorem, our first version of a theorem called the Master Theorem. (Later on we will develop some stronger forms of this theorem.) Theorem 5.1 Let a be an integer greater than or equal 1. Let c be a positive real number and d a nonnegative form aT (n/b) + nc T (n) = d then for n a power of b, 1. if logb a < c, T (n) = Θ(nc ), 2. if logb a = c, T (n) = Θ(nc log n), 3. if logb a > c, T (n) = Θ(nlogb a ). Proof: In this proof, we will set d = 1, so that the bottom level of the tree is equally well computed by the recursive step as by the base case. It is straightforward to extend the proof for the case when d = 1. Let’s think about the recursion tree for this recurrence. There will be logb n levels. At each level, the number of subproblems will be multiplied by a, and so the number of subproblems at level i will be ai . Each subproblem at level i is a problem of size (n/bi ). A subproblem of size n/bi requires (n/bi )c additional work and since there are ai problems on level i, the total number of units of work on level i is ai (n/bi )c = nc ai bci = nc a bc i to 1 and b be a real