Introduction to Algorithms Second Edition Instructor’s Manual 2nd phần 7 ppsx

Lecture Notes for Chapter 16: Greedy Algorithms Chapter 16 Introduction Similar to dynamic programming Used for optimization problems Idea: When we have a choice to make, make the one that looks best right now Make a locally optimal choice in hope of getting a globally optimal solution Greedy algorithms don’t always yield an optimal solution But sometimes they We’ll see a problem for which they Then we’ll look at some general characteristics of when greedy algorithms give optimal solutions [We not cover Huffman codes or matroids in these notes.] Activity selection n activities require exclusive use of a common resource For example, scheduling the use of a classroom Set of activities S = {a1 , , an } needs resource during period [si , f i ), which is a half-open interval, where si = start time and fi = Þnish time Goal: Select the largest possible set of nonoverlapping (mutually compatible) activities Note: Could have many other objectives: • • Schedule room for longest time Maximize income rental fees Example: S sorted by Þnish time: [Leave on board] i si fi 9 11 13 10 11 14 16 16-2 Lecture Notes for Chapter 16: Greedy Algorithms a5 a4 a2 a7 a1 a3 a9 a6 a8 10 11 12 13 14 15 16 Maximum-size mutually compatible set: {a1 , a3 , a6 , a8 } Not unique: also {a2 , a5 , a7 , a9 } Optimal substructure of activity selection Si j = {ak ∈ S : f i ≤ sk < f k ≤ s j } [Leave on board] = activities that start after Þnishes and Þnish before a j starts fi sk fk sj ak aj Activities in Si j are compatible with • • all activities that Þnish by fi , and all activities that start no earlier than sj To represent the entire problem, add Þctitious activities: a0 = [−∞, 0) an+1 = [∞, “∞ + 1”) We don’t care about −∞ in a0 or “∞ + 1” in an+1 Then S = S0,n+1 Range for Si j is ≤ i, j ≤ n + Assume that activities are sorted by monotonically increasing Þnish time: f ≤ f ≤ f ≤ · · · ≤ f n < f n+1 Then i ≥ j ⇒ Si j = ∅ • [Leave on board] If there exists ak ∈ Si j : f i ≤ sk < f k ≤ s j < f j ⇒ f i < f j • But i ≥ j ⇒ f i ≥ f j Contradiction So only need to worry about Si j with ≤ i < j ≤ n + All other Si j are ∅ Suppose that a solution to Si j includes ak Have subproblems: • • Sik (start after Þnishes, Þnish before ak starts) Skj (start after ak Þnishes, Þnish before a j starts) Lecture Notes for Chapter 16: Greedy Algorithms 16-3 Solution to Si j is (solution to Sik ) ∪ {ak } ∪ (solution to Skj ) Since ak is in neither subproblem, and the subproblems are disjoint, |solution to S| = |solution to Sik | + + |solution to Skj | If an optimal solution to Si j includes ak , then the solutions to Sik and Skj used within this solution must be optimal as well Use the usual cut-and-paste argument Let Ai j = optimal solution to Si j So Ai j = Aik ∪ {ak } ∪ Akj [leave on board] , assuming: • • Si j is nonempty, and we know ak Recursive solution to activity selection c[i, j ] = size of maximum-size subset of mutually compatible activities in Si j • i ≥ j ⇒ Si j = ∅ ⇒ c[i, j ] = If Si j = ∅, suppose we know that ak is in the subset Then c[i, j ] = c[i, k] + + c[k, j ] But of course we don’t know which k to use, and so ⎧ if Si j = ∅ , ⎨0 c[i, j ] = max {c[i, k] + c[k, j ] + 1} if Si j = ∅ ⎩ i i This procedure had an error: it worked only when j = n + It turns out that it was called only with j = n + 1, however To avoid this problem altogether, the procedure was changed to the following in the third printing.] R EC -ACTIVITY-S ELECTOR (s, f, i, n) m ←i +1 £ Find Þrst activity in Si,n+1 while m ≤ n and sm < f i m ← m + if m ≤ n then return {am } ∪ R EC -ACTIVITY-S ELECTOR (s, f, m, n) else return ∅ Initial call: R EC -ACTIVITY-S ELECTOR (s, f, 0, n) Idea: The while loop checks ai+1 , ai+2 , , an until it Þnds an activity am that is compatible with (need sm ≥ f i ) • • If the loop terminates because am is found (m ≤ n), then recursively solve Sm,n+1 , and return this solution, along with am If the loop never Þnds a compatible am (m > n), then just return empty set Go through example given earlier Should get {a1 , a4 , a8 , a11 } Time: (n)—each activity examined exactly once Can make this iterative It’s already almost tail recursive G REEDY-ACTIVITY-S ELECTOR (s, f, n) A ← {a1 } i ←1 for m ← to n if sm ≥ f i then A ← A ∪ {am } i ←m £ is most recent addition to A return A Go through example given earlier Should again get {a1 , a4 , a8 , a11 } Time: (n) Greedy strategy The choice that seems best at the moment is the one we go with What did we for activity selection? 16-6 Lecture Notes for Chapter 16: Greedy Algorithms Determine the optimal substructure Develop a recursive solution Prove that at any stage of recursion, one of the optimal choices is the greedy choice Therefore, it’s always safe to make the greedy choice Show that all but one of the subproblems resulting from the greedy choice are empty Develop a recursive greedy algorithm Convert it to an iterative algorithm At Þrst, it looked like dynamic programming Typically, we streamline these steps Develop the substructure with an eye toward • • making the greedy choice, leaving just one subproblem For activity selection, we showed that the greedy choice implied that in S j , only i i varied, and j was Þxed at n + We could have started out with a greedy algorithm in mind: ã ã Deịne Si = {ak ∈ S : f i ≤ sk } Then show that the greedy choice—Þrst am to Þnish in Si —combined with optimal solution to Sm ⇒ optimal solution to Si Typical streamlined steps: Cast the optimization problem as one in which we make a choice and are left with one subproblem to solve Prove that there’s always an optimal solution that makes the greedy choice, so that the greedy choice is always safe Show that greedy choice and optimal solution to subproblem ⇒ optimal solution to the problem No general way to tell if a greedy algorithm is optimal, but two key ingredients are greedy-choice property and optimal substructure Greedy-choice property A globally optimal solution can be arrived at by making a locally optimal (greedy) choice Dynamic programming: • • • Make a choice at each step Choice depends on knowing optimal solutions to subproblems Solve subproblems Þrst Solve bottom-up Lecture Notes for Chapter 16: Greedy Algorithms 16-7 Greedy: • • • Make a choice at each step Make the choice before solving the subproblems Solve top-down Typically show the greedy-choice property by what we did for activity selection: • • • Look at a globally optimal solution If it includes the greedy choice, done Else, modify it to include the greedy choice, yielding another solution that’s just as good Can get efÞciency gains from greedy-choice property • • Preprocess input to put it into greedy order Or, if dynamic data, use a priority queue Optimal substructure Just show that optimal solution to subproblem and greedy choice ⇒ optimal solution to problem Greedy vs dynamic programming The knapsack problem is a good example of the difference 0-1 knapsack problem: • • • • n items Item i is worth $vi , weighs wi pounds Find a most valuable subset of items with total weight ≤ W Have to either take an item or not take it—can’t take part of it Fractional knapsack problem: Like the 0-1 knapsack problem, but can take fraction of an item Both have optimal substructure But the fractional knapsack problem has the greedy-choice property, and the 0-1 knapsack problem does not To solve the fractional problem, rank items by value/weight: vi /wi Let vi /wi ≥ vi+1 /wi+1 for all i F RACTIONAL -K NAPSACK (v, w, W ) load ← i ←1 while load < W and i ≤ n if wi ≤ W − load then take all of item i else take (W − load)/wi of item i add what was taken to load i ←i +1 16-8 Lecture Notes for Chapter 16: Greedy Algorithms Time: O(n lg n) to sort, O(n) thereafter Greedy doesn’t work for the 0-1 knapsack problem Might get empty space, which lowers the average value per pound of the items taken i vi wi vi /wi 60 100 120 10 20 30 W = 50 Greedy solution: • • Take items and value = 160, weight = 30 Have 20 pounds of capacity left over Optimal solution: • • Take items and value = 220, weight = 50 No leftover capacity Solutions for Chapter 16: Greedy Algorithms Solution to Exercise 16.1-2 The proposed approach—selecting the last activity to start that is compatible with all previously selected activities—is really the greedy algorithm but starting from the end rather than the beginning Another way to look at it is as follows We are given a set S = {a1 , a2 , , an } of activities, where = [si , f i ), and we propose to Þnd an optimal solution by selecting the last activity to start that is compatible with all previously selected activities Instead, let us create a set S = {a1 , a2 , , an }, where = [ f i , si ) That is, is in reverse Clearly, a subset of {ai1 , ai2 , , aik } ⊆ S is mutually compatible if and only if the corresponding subset {ai1 , ai2 , , aik } ⊆ S is also mutually compatible Thus, an optimal solution for S maps directly to an optimal solution for S and vice versa The proposed approach of selecting the last activity to start that is compatible with all previously selected activities, when run on S, gives the same answer as the greedy algorithm from the text—selecting the Þrst activity to Þnish that is compatible with all previously selected activities—when run on S The solution that the proposed approach Þnds for S corresponds to the solution that the text’s greedy algorithm Þnds for S , and so it is optimal Solution to Exercise 16.1-3 Let S be the set of n activities The “obvious” solution of using G REEDY-ACTIVITY-S ELECTOR to Þnd a maximum-size set S1 of compatible activities from S for the Þrst lecture hall, then using it again to Þnd a maximum-size set S2 of compatible activities from S − S1 for the second hall, (and so on until all the activities are assigned), requires (n2 ) time in the worst case There is a better algorithm, however, whose asymptotic time is just the time needed to sort the activities by time—O(n lg n) time for arbitrary times, or possibly as fast as O(n) if the times are small integers The general idea is to go through the activities in order of start time, assigning each to any hall that is available at that time To this, move through the set 16-10 Solutions for Chapter 16: Greedy Algorithms of events consisting of activities starting and activities Þnishing, in order of event time Maintain two lists of lecture halls: Halls that are busy at the current eventtime t (because they have been assigned an activity i that started at s ≤ t but i won’t Þnish until fi > t) and halls that are free at time t (As in the activityselection problem in Section 16.1, we are assuming that activity time intervals are half open—i.e., that if si ≥ f j , then activities i and j are compatible.) When t is the start time of some activity, assign that activity to a free hall and move the hall from the free list to the busy list When t is the Þnish time of some activity, move the activity’s hall from the busy list to the free list (The activity is certainly in some hall, because the event times are processed in order and the activity must have started before its Þnish time t, hence must have been assigned to a hall.) To avoid using more halls than necessary, always pick a hall that has already had an activity assigned to it, if possible, before picking a never-used hall (This can be done by always working at the front of the free-halls list—putting freed halls onto the front of the list and taking halls from the front of the list—so that a new hall doesn’t come to the front and get chosen if there are previously-used halls.) This guarantees that the algorithm uses as few lecture halls as possible: The algorithm will terminate with a schedule requiring m ≤ n lecture halls Let activity i be the Þrst activity scheduled in lecture hall m The reason that i was put in the mth lecture hall is that the Þrst m − lecture halls were busy at time si So at this time there are m activities occurring simultaneously Therefore any schedule must use at least m lecture halls, so the schedule returned by the algorithm is optimal Run time: • • Sort the 2n activity-starts/activity-ends events (In the sorted order, an activityending event should precede an activity-starting event that is at the same time.) O(n lg n) time for arbitrary times, possibly O(n) if the times are restricted (e.g., to small integers) Process the events in O(n) time: Scan the 2n events, doing O(1) work for each (moving a hall from one list to the other and possibly associating an activity with it) Total: O(n + time to sort) [The idea of this algorithm is related to the rectangle-overlap algorithm in Exercise 14.3-7.] Solution to Exercise 16.1-4 • For the approach of selecting the activity of least duration from those that are compatible with previously selected activities: i si fi duration 3 2 3 This approach selects just {a2 }, but the optimal solution selects {a1 , a3 } Lecture Notes for Chapter 17: Amortized Analysis 17-9 Potential method (T ) = · num[T ] − size[T ] • • • • Initially, num = size = ⇒ = Just after expansion, size = · num ⇒ Just before expansion, size = num ⇒ pay for moving all items Need ≥ 0, always = = num ⇒ have enough potential to Always have size ≥ num ≥ · size ⇒ 2 · num ≥ size ⇒ ≥ Amortized cost of ith operation: numi sizei i = num after ith operation , = size after ith operation , = after ith operation • If no expansion: sizei = sizei−1 , numi = numi−1 +1 , ci = Then we have ci = ci + i − i−1 = + (2 · numi − sizei ) − (2 · numi−1 − sizei−1 ) = + (2 · numi − sizei ) − (2(numi −1) − sizei ) = 1+2 = • If expansion: sizei = · sizei−1 , sizei−1 = numi−1 = numi −1 , ci = numi−1 +1 = numi Then we have ci = ci + i + i−1 = numi + (2 · numi − sizei ) − (2 · numi−1 − sizei−1 ) = numi + (2 · numi −2(numi −1)) − (2(numi −1) − (numi −1)) = numi + − (numi −1) = 17-10 Lecture Notes for Chapter 17: Amortized Analysis 32 sizei 24 numi 16 Φi 0 16 24 32 i Expansion and contraction When α drops too low, contract the table • Allocate a new, smaller one • Copy all items Still want • α bounded from below by a constant, • amortized cost per operation = O(1) Measure cost in terms of elementary insertions and deletions “Obvious strategy”: • Double size when inserting into a full table (when α = 1, so that after insertion α would become > 1) • Halve size when deletion would make table less than half full (when α = 1/2, so that after deletion α would become < 1/2) • Then always have 1/2 ≤ α ≤ • Suppose we Þll table Then insert ⇒ double deletes ⇒ halve inserts ⇒ double deletes ⇒ halve ··· Not performing enough operations after expansion or contraction to pay for the next one Simple solution: Double as before: when inserting with α = ⇒ after doubling, α = 1/2 • Halve size when deleting with α = 1/4 ⇒ after halving, α = 1/2 • Thus, immediately after either expansion or contraction, have α = 1/2 • Always have 1/4 ≤ α ≤ • Lecture Notes for Chapter 17: Amortized Analysis 17-11 Intuition: • • • • Want to make sure that we perform enough operations between consecutive expansions/contractions to pay for the change in table size Need to delete half the items before contraction Need to double number of items before expansion Either way, number of operations between expansions/contractions is at least a constant fraction of number of items copied (T ) = · num[T ] − size[T ] if α ≥ 1/2 , size[T ]/2 − num[T ] if α < 1/2 T empty ⇒ = α ≥ 1/2 ⇒ num ≥ α < 1/2 ⇒ num < Intuition: • • • • • • 2 · size ⇒ · num ≥ size ⇒ · size ⇒ ≥ ≥ measures how far from α = 1/2 we are α = 1/2 ⇒ = · num −2 · num = α = ⇒ = · num − num = num α = 1/4 ⇒ = size /2 − num = · num /2 − num = num Therefore, when we double or halve, have enough potential to pay for moving all num items Potential increases linearly between α = 1/2 and α = 1, and it also increases linearly between α = 1/2 and α = 1/4 Since α has different distances to go to get to or 1/4, starting from 1/2, rate of increase of differs • • For α to go from 1/2 to 1, num increases from size /2 to size, for a total increase of size /2 increases from to size Thus, needs to increase by for each item inserted That’s why there’s a coefÞcient of on the num[T ] term in the formula for when α ≥ 1/2 For α to go from 1/2 to 1/4, num decreases from size /2 to size /4, for a total decrease of size /4 increases from to size /4 Thus, needs to increase by for each item deleted That’s why there’s a coefÞcient of −1 on the num[T ] term in the formula for when α < 1/2 Amortized costs: more cases • • • insert, delete α ≥ 1/2, α < 1/2 (use αi , since α can vary a lot) size does/doesn’t change Insert: • • αi−1 ≥ 1/2, same analysis as before ⇒ ci = αi−1 < 1/2 ⇒ no expansion (only occurs when αi−1 = 1) 17-12 Lecture Notes for Chapter 17: Amortized Analysis • If αi−1 ci = = = = < 1/2 and αi < 1/2: ci + i + i−1 + (sizei /2 − numi ) − (sizei−1 /2 − numi−1 ) + (sizei /2 − numi ) − (sizei /2 − (numi −1)) • If αi−1 < 1/2 and αi ≥ 1/2: ci = + (2 · numi − sizei ) − (sizei−1 /2 − numi−1 ) = + (2(numi−1 +1) − sizei−1 ) − (sizei−1 /2 − numi−1 ) = · numi−1 − · sizei−1 +3 = · αi−1 sizei−1 − · sizei−1 +3 3 · sizei−1 − · sizei−1 +3 < 2 = Therefore, amortized cost of insert is < Delete: • If αi−1 < 1/2, then αi < 1/2 • If no contraction: ci = + (sizei /2 − numi ) − (sizei−1 /2 − numi−1 ) = + (sizei /2 − numi ) − (sizei /2 − (numi +1)) = • If contraction: ci = (numi +1) + (sizei /2 − numi ) − (sizei−1 /2 − numi−1 ) move + delete [sizei /2 = sizei−1 /4 = numi−1 = numi +1] = (numi +1) + ((numi +1) − numi ) − ((2 · numi +2) − (numi +1)) = • If αi−1 ≥ 1/2, then no contraction • If αi ≥ 1/2: ci = + (2 · numi − sizei ) − (2 · numi−1 − sizei−1 ) = + (2 · numi − sizei ) − (2 · numi +2 − sizei ) = −1 • If αi < 1/2, since αi−1 ≥ 1/2, have 1 numi = numi−1 −1 ≥ · sizei−1 −1 = · sizei −1 2 Lecture Notes for Chapter 17: Amortized Analysis Thus, ci = + (sizei /2 − numi ) − (2 · numi−1 − sizei−1 ) = + (sizei /2 − numi ) − (2 · numi +2 − sizei ) = −1 + · sizei −3 · numi · sizei −1 ≤ −1 + · sizei −3 2 = Therefore, amortized cost of delete is ≤ 17-13 Solutions for Chapter 17: Amortized Analysis Solution to Exercise 17.1-3 Let ci = cost of ith operation ci = i if i is an exact power of , otherwise Operation 10 Cost 1 1 n operations cost lg n n ci ≤ n + i=1 j = n + (2n − 1) < 3n j =0 (Note: Ignoring ßoor in upper bound of j ) Average cost of operation = Total cost < # operations By aggregate analysis, the amortized cost per operation = O(1) Solution to Exercise 17.2-1 [We assume that the only way in which C OPY is invoked is automatically, after every sequence of k P USH and P OP operations.] Solutions for Chapter 17: Amortized Analysis 17-15 Charge $2 for each P USH and P OP operation and $0 for each C OPY When we call P USH, we use $1 to pay for the operation, and we store the other $1 on the item pushed When we call P OP, we again use $1 to pay for the operation, and we store the other $1 in the stack itself Because the stack size never exceeds k, the actual cost of a C OPY operation is at most $k, which is paid by the $k found in the items in the stack and the stack itself Since there are k P USH and P OP operations between two consecutive C OPY operations, there are $k of credit stored, either on individual items (from P USH operations) or in the stack itself (from P OP operations) by the time a C OPY occurs Since the amortized cost of each operation is O(1) and the amount of credit never goes negative, the total cost of n operations is O(n) Solution to Exercise 17.2-2 Let ci = cost of ith operation ci = i if i is an exact power of , otherwise Charge each operation $3 (amortized cost ci ) • • If i is not an exact power of 2, pay $1, and store $2 as credit If i is an exact power of 2, pay $i, using stored credit Operation 10 Cost 3 3 3 3 3 Actual cost 1 1 Credit remaining 10 n ci = 3n Since the amortized cost is $3 per operation, i=1 n ci < 3n We know from Exercise 17.1-3 that i=1 n n ci ≥ Then we have i=1 ci ⇒ credit = amortized cost − actual cost ≥ i=1 Since the amortized cost of each operation is O(1), and the amount of credit never goes negative, the total cost of n operations is O(n) 17-16 Solutions for Chapter 17: Amortized Analysis Solution to Exercise 17.2-3 We introduce a new Þeld max[A] to hold the index of the high-order in A Initially, max[A] is set to −1, since the low-order bit of A is at index 0, and there are initially no 1’s in A The value of max[A] is updated as appropriate when the counter is incremented or reset, and we use this value to limit how much of A must be looked at to reset it By controlling the cost of R ESET in this way, we can limit it to an amount that can be covered by credit from earlier I NCREMENTs I NCREMENT ( A) i ←0 while i < length[A] and A[i] = A[i] ← i ←i +1 if i < length[A] then A[i] ← £ Additions to book’s I NCREMENT start here if i > max[A] then max[A] ← i else max[A] ← −1 R ESET ( A) for i ← to max[A] A[i] ← max[A] ← −1 As for the counter in the book, we assume that it costs $1 to ßip a bit In addition, we assume it costs $1 to update max[A] Setting and resetting of bits by I NCREMENT will work exactly as for the original counter in the book: $1 will pay to set one bit to 1; $1 will be placed on the bit that is set to as credit; the credit on each bit will pay to reset the bit during incrementing In addition, we’ll use $1 to pay to update max, and if max increases, we’ll place an additional $1 of credit on the new high-order (If max doesn’t increase, we can just waste that $1—it won’t be needed.) Since R ESET manipulates bits at positions only up to max[A], and since each bit up to there must have become the high-order at some time before the high-order got up to max[A], every bit seen by R ESET has $1 of credit on it So the zeroing of bits of A by R ESET can be completely paid for by the credit stored on the bits We just need $1 to pay for resetting max Thus charging $4 for each I NCREMENT and $1 for each R ESET is sufÞcient, so the sequence of n I NCREMENT and R ESET operations takes O(n) time Solutions for Chapter 17: Amortized Analysis 17-17 Solution to Exercise 17.3-3 Let Di be the heap after the ith operation, and let Di consist of ni elements Also, let k be a constant such that each I NSERT or E XTRACT-M IN operation takes at most k ln n time, where n = max(ni−1 , n i ) (We don’t want to worry about taking the log of 0, and at least one of ni−1 and n i is at least We’ll see later why we use the natural log.) DeÞne (Di ) = kn i ln n i if n i = , if n i > This function exhibits the characteristics we like in a potential function: if we start with an empty heap, then (D0 ) = 0, and we always maintain that (Di ) ≥ Before proving that we achieve the desired amortized times, we show that if n ≥ 2, n then n ln n−1 ≤ We have n ln n n−1 n−1 n = ln + n−1 = n ln + ≤ ln e n−1 n (since + x ≤ ex for all real x) n n−1 = ln e n = n−1 ≤ 2, n assuming that n ≥ (The equation ln e n−1 = n n−1 is why we use the natural log.) If the ith operation is an I NSERT, then ni = n i−1 + If the ith operation inserts into an empty heap, then ni = 1, n i−1 = 0, and the amortized cost is ci = ci + (Di ) − (Di−1 ) ≤ k ln + k · ln − = If the ith operation inserts into a nonempty heap, then ni = n i−1 + 1, and the amortized cost is ci = ci + (Di ) − (Di−1 ) ≤ k ln n i + kn i ln n i − kn i−1 ln n i−1 = k ln n i + kn i ln n i − k(n i − 1) ln(n i − 1) = k ln n i + kn i ln n i − kn i ln(n i − 1) + k ln(n i − 1) ni < 2k ln n i + kn i ln ni − ≤ 2k ln n i + 2k = O(lg n i ) If the ith operation is an E XTRACT-M IN, then ni = n i−1 − If the ith operation extracts the one and only heap item, then ni = 0, n i−1 = 1, and the amortized cost 17-18 Solutions for Chapter 17: Amortized Analysis is ci = ci + (Di ) − (Di−1 ) ≤ k ln + − k · ln = If the ith operation extracts from a heap with more than item, then ni = n i−1 − and n i−1 ≥ 2, and the amortized cost is ci = ci + (Di ) − (Di−1 ) ≤ k ln n i−1 + kn i ln n i − kn i−1 ln n i−1 = k ln n i−1 + k(n i−1 − 1) ln(n i−1 − 1) − kn i−1 ln n i−1 = k ln n i−1 + kn i−1 ln(n i−1 − 1) − k ln(n i−1 − 1) − kn i−1 ln n i−1 n i−1 n i−1 − + kn i−1 ln = k ln n i−1 − n i−1 n i−1 + kn i−1 ln < k ln n i−1 − n i−1 = k ln n i−1 − ≤ k ln (since ni−1 ≥ 2) = O(1) A slightly different potential function—which may be easier to work with—is as follows For each node x in the heap, let di (x) be the depth of x in Di DeÞne k(di (x) + 1) (Di ) = x∈Di = k ni + di (x) , x∈Di where k is deÞned as before Initially, the heap has no items, which means that the sum is over an empty set, and so (D0 ) = We always have (Di ) ≥ 0, as required Observe that after an I NSERT, the sum changes only by an amount equal to the depth of the new last node of the heap, which is lg n i Thus, the change in potential due to an I NSERT is k(1 + lg n i ), and so the amortized cost is O(lg n i ) + O(lg n i ) = O(lg n i ) = O(lg n) After an E XTRACT-M IN, the sum changes by the negative of the depth of the old last node in the heap, and so the potential decreases by k(1 + lg n i−1 ) The amortized cost is at most k lg ni−1 − k(1 + lg n i−1 ) = O(1) Solution to Problem 17-2 a The S EARCH operation can be performed by searching each of the individually sorted arrays Since all the individual arrays are sorted, searching one of them using a binary search algorithm takes O(lg m) time, where m is the size of the array In an unsuccessful search, the time is (lg m) In the worst case, we may Solutions for Chapter 17: Amortized Analysis 17-19 assume that all the arrays A0 , A1 , , Ak−1 are full, k = lg(n + 1) , and we perform an unsuccessful search The total time taken is T (n) = (lg 2k−1 + lg 2k−2 + · · · + lg 21 + lg 20 ) = ((k − 1) + (k − 2) + · · · + + 0) = (k(k − 1)/2) = ( lg(n + 1) ( lg(n + 1) − 1)/2) = (lg2 n) Thus, the worst-case running time is (lg2 n) b We create a new sorted array of size containing the new element to be inserted If array A0 (which has size 1) is empty, then we replace A0 with the new sorted array Otherwise, we merge sort the two arrays into another sorted array of size If A1 is empty, then we replace A1 with the new array; otherwise we merge sort the arrays as before and continue Since array Ai is of size 2i , if we merge sort two arrays of size 2i each, we obtain one of size 2i+1 , which is the size of Ai+1 Thus, this method will result in another list of arrays in the same structure that we had before Let us analyze its worst-case running time We will assume that merge sort takes 2m time to merge two sorted lists of size m each If all the arrays A0 , A1 , , Ak−2 are full, then the running time to Þll array Ak−1 would be T (n) = (20 + 21 + · · · + 2k−2 ) = 2(2k−1 − 1) = 2k − = (n) Therefore, the worst-case time to insert an element into this data structure is (n) However, let us now analyze the amortized running time Using the aggregate method, we compute the total cost of a sequence of n inserts, starting with the empty data structure Let r be the position of the rightmost in the binary representation nk−1 , n k−2 , , n of n, so that n j = for j = 0, 1, , r − The cost of an insertion when n items have already been inserted is r−1 · j = O(2r ) j =0 Furthermore, r = half the time, r = a quarter of the time, and so on There are at most n/2r insertions for each value of r The total cost of the n operations is therefore bounded by lg(n+1) O r=0 n 2r 2r = O(n lg n) The amortized cost per I NSERT operation, therefore is O(lg n) We can also use the accounting method to analyze the running time We can charge $k to insert an element $1 pays for the insertion, and we put $(k − 1) on the inserted item to pay for it being involved in merges later on Each time it is merged, it moves to a higher-indexed array, i.e., from Ai to Ai+1 It can 17-20 Solutions for Chapter 17: Amortized Analysis move to a higher-indexed array at most k − times, and so the $(k − 1) on the item sufÞces to pay for all the times it will ever be involved in merges Since k = (lg n), we have an amortized cost of (lg n) per insertion c D ELETE (x) will be implemented as follows: Find the smallest j for which the array Aj with j elements is full Let y be the last element of A j Let x be in the array Ai If necessary, Þnd which array this is by using the search procedure Remove x from Ai and put y into Ai Then move y to its correct place in Ai Divide A j (which now has j − elements left): The Þrst element goes into array A0 , the next elements go into array A1 , the next elements go into array A2 , and so forth Mark array A j as empty The new arrays are created already sorted The cost of D ELETE is (n) in the worst case, where i = k − and j = k − 2: (lg n) to Þnd A j , (lg2 n) to Þnd Ai , (2i ) = (n) to put y in its correct place in array Ai , and (2 j ) = (n) to divide array A j The following sequence of n operations, where n/3 is a power of 2, yields an amortized cost that is no better: perform n/3 I NSERT operations, followed by n/3 pairs of D ELETE and I NSERT It costs O(n lg n) to the Þrst n/3 I NSERT operations This creates a single full array Each subsequent D ELETE/I NSERT pair costs (n) for the D ELETE to divide the full array and another (n) for the I NSERT to recombine it The total is then (n2 ), or (n) per operation Solution to Problem 17-4 a For RB-I NSERT, consider a complete red-black tree in which the colors alternate between levels That is, the root is black, the children of the root are red, the grandchildren of the root are black, the great-grandchildren of the root are red, and so on When a node is inserted as a red child of one of the red leaves, then case of RB-I NSERT-F IXUP occurs (lg(n + 1))/2 times, so that there are (lg n) color changes to Þx the colors of nodes on the path from the inserted node to the root For RB-D ELETE, consider a complete red-black tree in which all nodes are black If a leaf is deleted, then the double blackness will be pushed all the way up to the root, with a color change at each level (case of RB-D ELETE -F IXUP ), for a total of (lg n) color changes b All cases except for case of RB-I NSERT-F IXUP and case of RB-D ELETE F IXUP are terminating c Case of RB-I NSERT-F IXUP reduces the number of red nodes by As Figure 13.5 shows, node z’s parent and uncle change from red to black, and z’s grandparent changes from black to red Hence, (T ) = (T ) − d Lines 1–16 of RB-I NSERT cause one node insertion and a unit increase in potential The nonterminating case of RB-I NSERT-F IXUP (Case 1) makes three Solutions for Chapter 17: Amortized Analysis 17-21 color changes and decreases the potential by The terminating cases of RBI NSERT-F IXUP (cases and 3) cause one rotation each and not affect the potential (Although case makes color changes, the potential does not change As Figure 13.6 shows, node z’s parent changes from red to black, and z’s grandparent changes from black to red.) e The number of structural modiÞcations and amount of potential change resulting from lines 1–16 of RB-I NSERT and from the terminating cases of RBI NSERT-F IXUP are O(1), and so the amortized number of structural modiÞcations of these parts is O(1) The nonterminating case of RB-I NSERT-F IXUP may repeat O(lg n) times, but its amortized number of structural modiÞcations is 0, since by our assumption the unit decrease in the potential pays for the structural modiÞcations needed Therefore, the amortized number of structural modiÞcations performed by RB-I NSERT is O(1) f From Figure 13.5, we see that case of RB-I NSERT-F IXUP makes the following changes to the tree: • • • Changes a black node with two red children (node C) to a red node, resulting in a potential change of −2 Changes a red node (node A in part (a) and node B in part (b)) to a black node with one red child, resulting in no potential change Changes a red node (node D) to a black node with no red children, resulting in a potential change of The total change in potential is −1, which pays for the structural modiÞcations performed, and thus the amortized number of structural modiÞcations in case (the nonterminating case) is The terminating cases of RB-I NSERT-F IXUP cause O(1) structural changes Because w(v) is based solely on node colors and the number of color changes caused by terminating cases is O(1), the change in potential in terminating cases is O(1) Hence, the amortized number of structural modiÞcations in the terminating cases is O(1) The overall amortized number of structural modiÞcations in RB-I NSERT, therefore, is O(1) g Figure 13.7 shows that case of RB-D ELETE -F IXUP makes the following changes to the tree: • • • Changes a black node with no red children (node D) to a red node, resulting in a potential change of −1 If B is red, then it loses a black child, with no effect on potential If B is black, then it goes from having no red children to having one red child, resulting in a potential change of −1 The total change in potential is either −1 or −2, depending on the color of B In either case, one unit of potential pays for the structural modiÞcations performed, and thus the amortized number of structural modiÞcations in case (the nonterminating case) is at most The terminating cases of RB-D ELETE cause O(1) structural changes Because w(v) is based solely on node colors and the number of color changes caused by terminating cases is O(1), the change in potential in terminating cases is O(1) Hence, the amortized number of structural changes in the terminating cases is O(1) The overall amortized number of structural modiÞcations in RB-D ELETE -F IXUP , therefore, is O(1) 17-22 Solutions for Chapter 17: Amortized Analysis h Since the amortized number structural modiÞcation in each operation is O(1), the actual number of structural modiÞcations for any sequence of m RBI NSERT and RB-D ELETE operations on an initially empty red-black tree is O(m) in the worst case Lecture Notes for Chapter 21: Data Structures for Disjoint Sets Chapter 21 overview Disjoint-set data structures • • • Also known as “union Þnd.” Maintain collection S = {S1, , Sk } of disjoint dynamic (changing over time) sets Each set is identiÞed by a representative, which is some member of the set Doesn’t matter which member is the representative, as long as if we ask for the representative twice without modifying the set, we get the same answer both times [We not include notes for the proof of running time of the disjoint-set forest implementation, which is covered in Section 21.4.] Operations • • M AKE -S ET (x): make a new set Si = {x}, and add Si to S U NION (x, y): if x ∈ Sx , y ∈ S y , then S ← S − Sx − S y ∪ {Sx ∪ S y } • • • Representative of new set is any member of Sx ∪ S y , often the representative of one of Sx and Sy Destroys Sx and Sy (since sets must be disjoint) F IND -S ET (x): return representative of set containing x Analysis in terms of: • • n = # of elements = # of M AKE -S ET operations, m = total # of operations ... Greedy Algorithms 16-3 Solution to Si j is (solution to Sik ) ∪ {ak } ∪ (solution to Skj ) Since ak is in neither subproblem, and the subproblems are disjoint, |solution to S| = |solution to Sik... sorting A and B into monotonically increasing order works as well 16-14 Solutions for Chapter 16: Greedy Algorithms Solution to Exercise 16.4-2 We need to show three things to prove that (S,... negative Therefore, total amortized cost, = O(n), is an upper bound on total actual cost Lecture Notes for Chapter 17: Amortized Analysis 17- 5 Binary counter Charge $2 to set a bit to • • • • $1 pays