In order to become the master, the politician poses as the servant.
Charles de Gaulle (1890–1970) In the remainder of this section, we’ll turn to a more formulaic method, called the Master Method,of solving recurrence relations that have a certain form: in analyzing algorithms, we will frequently encounter recurrences that look like
T(n) =aT nb+cãnk, for four constantsa≥1,b>1,c>0, andk≥0.
Why do these recurrences come up frequently? Consider a recursive algorithm that has the following structure: if the input is small—say,n = 1—then we compute the solution directly; otherwise, to solve an instance of sizen:
• we makeadifferent recursive calls on inputs of sizenb; and
• to construct the smaller instances and then to reconstruct the solution to the given instance from the recursive solutions, we spend Θ(nk) time.
(These algorithms are usually calleddivide-and-conquer algorithms:they “divide” their input intoapieces, and then recursively “conquer” those subproblems.) To be precise, the recurrence often has ceilings and floors as part of its recursive calls, but for now assume thatnis exact power ofb, so that the floors and ceilings don’t matter.
Here are a few examples of recursive algorithms with recurrences of this form:
Example 6.29 (Binary Search)
We spendc= Θ(1) time to compare the sought element to the middle of the range; we then make one recursive call to search for the element in the appropriate half of the array. Ifnis an exact power of two, then the recurrence is
T(n) =T(n2) +c.
(Soa= 1,b= 2, andk= 0, becausec=cã1 =cãn0.)
Example 6.30 (Merge Sort)
We spend Θ(1) time to divide the array in half. We make two recursive calls on the left and right subarrays, and then spend Θ(n) time to merge the resulting sorted subarrays into a single sorted array. Ifnis an exact power of two, then the recurrence is
T(n) = 2T(n2) +cãn.
(Soa= 2,b= 2, andk= 1.)
n
nb n
b n
b
1 2
ã ã ã
a
bn2 n
b2 n
b2 n
b2 n
b2 n
b2
1 : 1 1 : 2 1 :a a: 1 a: 2 a:a
ã ã ã
ã ã ã ã ã ã ã ã ã ã ã ã ã ã ã ã ã ã
b 1 1 ã ã ã 1
b 1 1 ã ã ã 1
ã ã ã b
1 1 ã ã ã 1
. .. ...
ã ã ã ã ã ã
Figure 6.44: The recursion tree for a recurrence relationT(n) = aT(nb) +cãnk, of the master method’s form. Assume that nis an exact power ofb.
6.5.1 The Master Method: Some Intuition
TheMaster Methodis a technique that allows us to solve any recurrence relation of the formT(n) = aT(nb) +cãnkvery easily. The Master Method is based on examining the recursion tree for this recurrence (see Figure 6.44), and theMaster Theorem(Theo- rem 6.10) that describes the total amount of work represented by this tree.
Here’s the intuition for the Master Method. Let’s think about theith level of the recursion tree (again, see Figure 6.44)—in other words, the work done by the recursive calls that areilevels beneath the root of the recursion tree. Observe the following:
There are aidifferent calls at level i. There is 1 =a0call at the 0th level, thena=a1calls at 1st level, thena2calls at the 2nd level, and so forth.
Each of the the calls at the ith level operates on an input of size bni. The input size isn1 =nat the 0th level, then nb at the 1st level, thenbn2 at the 2nd, and so forth.
Thus the total amount of work in the ith level of the tree is aiãcã(bni)k. Or, simplifying, the total work at this level iscnkã(bak)i.
Thus the total amount of work contained within the entire tree is
∑i
cnkãbaki
=cnkã∑
i
a bk
i
. (∗)
(We’ll worry about the bounds on the summation later.)
Note that (∗) expresses the total work in the recursion tree as a geometric sum∑iri, in which the ratio between terms is given byr := bak. (See Section 5.2.2.) As with any geometric sum, the critical question is how the ratio compares to 1: ifr <1, then the terms of the sum are getting smaller and smaller asiincreases; ifr>1, then the terms of the sum are getting bigger and bigger asiincreases. (And ifr= 1, then each term is simply equal to 1.)
The Master Theorem has three cases, each of which corresponds to one of these three natural cases for the summation in (∗): its termsincrease exponentiallywithi, its
termsdecrease exponentiallywithi, or its terms areconstantwith respect toi. In these cases, respectively, almost all of the work is done at the leaves of the tree; almost all of the work is done at the root of the tree; or the work is spread evenly across the levels of the tree. (Here “almost all the work” means “a constant fraction of the work,” which means that the total work in the tree is asymptotically equivalent to the work done solely at the root or at the leaves.)
A trio of examples
Before we prove the general theorem, we’ll solve a few recurrences that illustrate the cases of the Master Method, and then we’ll prove the result in general. The three example recurrences are
T(n) = 2T(n2) + 1 T(n) = 2T(n2) +n andT(n) = 2T(n2) +n2,
all withT(1) = 1. Figure 6.45 shows the recursion trees for these recurrences.
1 call(s), 1 orn1or (n1)2work each
2 call(s), 1 orn2or (n2)2work each
4 call(s), 1 orn4or (n4)2work each ...
2icall(s), 1 or2ni or (2ni)2work each ...
ncall(s), 1 or 1 or 1 work each n
n2 n
2
n4 n
4 n
4 n
4
...
1 1
2 2
ã ã ã ã ã ã
. .. ...
1 1
Figure 6.45: The recursion trees for three different recurrences:T(n) = 2T(n2) +f(n), for f(n)∈1,n,n2 . The annotation in each row of the tree shows both the number of calls at that level of the tree, plus the additional work done by each call at that level.
In each of these recurrences, we divide the input by two at every level of the recur- sion. Thus, the total depth of the recursion tree is log2n. (Assumenis an exact power of two.) In the recursion tree for any one of these recurrences, consider theith level of the tree beneath the root. (The root of the recursion tree has depth 0.) We have divided nby 2 a total ofitimes, and thus the input size at that level is 2ni. Furthermore, there are 2idifferent calls at theith level of the tree.
Solving the three recurrences
To solve each recurrence, we will sum the total amount of work generated at each level of the tree. The recursion trees for each of these three recurrences are shown in Figures 6.46, 6.47, and 6.48.
= 1ã1 = 1
= 2ã1 = 2
= 4ã1 = 4 ...
= 2iã1 = 2i ...
= 2log2nã1 =n n
n2 n
2
n4 n
4 n
4 n
4
...
1 1
2 2
ã ã ã ã ã ã
. .. ...
1 1
Figure 6.46: The recursion tree for T(n) = 2T(n2) + 1, with the “row- wise” sums of work. The work at each level is twice the work at the level above it; thus the work is increasing exponentially at each level of the tree.
= 1ãn1 =n
= 2ãn2 =n
= 4ãn4 =n ...
= 2iã2ni =n ...
= 2log2nã1 =n n
n2 n
2
n4 n
4 n
4 n
4
...
1 1
2 2
ã ã ã ã ã ã
. .. ...
1 1
Figure 6.47: The recursion tree for T(n) = 2T(n2) +n.
The work at each level is exactlyn;
thus the work is constant across the levels of the tree.
= 1ã(n1)2=n12
= 2ã(n2)2=n22
= 4ã(n4)2=n42 ...
= 2iã(2ni)2=n22i
...
= 2log2nã1 =nn2
n
n2 n
2
n4 n
4 n
4 n
4
...
1 1
2 2
ã ã ã ã ã ã
. .. ...
1 1
Figure 6.48: The recursion tree for T(n) = 2T(n2) +n2. The work at each level is half of the work at the level above it; thus the work is decreasing exponentially at each level of the tree.
Example 6.31 (SolvingT(n) = 2T(n2) + 1)
Figure 6.46 shows the recursion tree for this recurrence. There are 2idifferent calls at theith level, each of which is on an input of size 2ni—and we do 1 unit of work for each of these 2icalls. Thus the total amount of work at leveliis 2i. The total amount of work in the entire tree is therefore
T(n) =log∑2n
i=0 2i= 21+log2n−1
2−1 = 2ã2log2n= 2n by Theorem 5.2. And, indeed,T(n) = Θ(n).
Example 6.32 (SolvingT(n) = 2T(n2) +n)
Figure 6.47 shows the recursion tree. There are 2icalls at theith level of the recursion tree, on inputs of size2ni. We do2ni units of work at each call, so the total work at the ith level is 2iã(2ni) = n. Note that the amount of work at leveliis independent of the leveli. The total amount of work in the tree is therefore
T(n) =log∑2n
i=0 n
work at level #i|{z}
=nã
log2n
∑i=0 1 =n(1 + log2n) = Θ(nlogn).
Example 6.33 (SolvingT(n) = 2T(n2) +n2)
Figure 6.48 shows the recursion tree. There are 2icalls at theith level of the tree, and we do (2ni)2work at each call at this level. Thus the work represented by theith row of the recursion tree is (2ni)2ã2i= n22i. The total amount of work in the tree is therefore
T(n) =log∑2n
i=0 (12)in2=n2ã
log2n
∑i=0 (12)i.
Notice that∑logi=02n(12)i = 1 + 12+14+ã ã ã+2log21 n, which is certainly at least 1. But, by the fact that 1 +12+14+ . . . +21ℓ < 2 (see Theorem 5.2), we also know∑logi=02n(12)i ≤ 2.
Thereforen2≤T(n)≤2n2, which allows us to conclude thatT(n) = Θ(n2).
6.5.2 The Master Method: The Formal Statement and a Proof
Examples 6.31, 6.32, and 6.33 were designed to build the necessary intuition about the three different cases of the master method: work increases exponentially across levels of the recursion tree; work stays constant across levels; or work decreases expo- nentially across levels. Precisely the same intuition will yield the proof of the Master Theorem. Here is the formal statement of the Master Theorem, which generalizes the idea of these examples to all recurrences of the formT(n) =aT(nb) +cnk: