CuuDuongThanCong.com CuuDuongThanCong.com Algorithm Design and Applications Michael T Goodrich Department of Information and Computer Science University of California, Irvine Roberto Tamassia Department of Computer Science Brown University CuuDuongThanCong.com CuuDuongThanCong.com iii CuuDuongThanCong.com To Karen, Paul, Anna, and Jack – Michael T Goodrich To Isabel – Roberto Tamassia CuuDuongThanCong.com Contents Preface xi Algorithm Analysis 1.1 1.2 1.3 1.4 1.5 Analyzing Algorithms A Quick Mathematical Review A Case Study in Algorithm Analysis Amortization Exercises Part I: Data Structures Basic Data Structures 2.1 2.2 2.3 2.4 51 Stacks and Queues Lists Trees Exercises Searches and Updates Range Queries Index-Based Searching Randomly Constructed Search Trees Exercises Binary Search Trees 3.1 3.2 3.3 3.4 3.5 Ranks and Rotations AVL Trees Red-Black Trees Weak AVL Trees Splay Trees Exercises Priority Queues PQ-Sort, Selection-Sort, and Insertion-Sort Heaps Heap-Sort Extending Priority Queues Exercises v CuuDuongThanCong.com 91 101 104 107 110 115 Priority Queues and Heaps 5.1 5.2 5.3 5.4 5.5 5.6 53 60 68 84 89 Balanced Binary Search Trees 4.1 4.2 4.3 4.4 4.5 4.6 19 29 34 42 117 120 126 130 139 149 155 157 158 163 174 179 182 Contents vi Hash Tables 6.1 6.2 6.3 6.4 6.5 6.6 187 Maps Hash Functions Handling Collisions and Rehashing Cuckoo Hashing Universal Hashing Exercises Union-Find Structures 7.1 7.2 7.3 7.4 219 Union-Find and Its Applications A List-Based Implementation A Tree-Based Implementation Exercises Part II: 241 Merge-Sort Quick-Sort A Lower Bound on Comparison-Based Sorting Exercises Fast Sorting and Selection 9.1 9.2 9.3 9.4 Bucket-Sort and Radix-Sort Selection Weighted Medians Exercises Part III: CuuDuongThanCong.com 267 270 276 279 Fundamental Techniques 283 The Fractional Knapsack Problem Task Scheduling Text Compression and Huffman Coding Exercises 11 Divide-and-Conquer 11.1 11.2 11.3 11.4 11.5 243 250 257 259 265 10 The Greedy Method 10.1 10.2 10.3 10.4 221 225 228 236 Sorting and Selection Merge-Sort and Quick-Sort 8.1 8.2 8.3 8.4 189 192 198 206 212 215 Recurrences and the Master Theorem Integer Multiplication Matrix Multiplication The Maxima-Set Problem Exercises 286 289 292 298 303 305 313 315 317 319 Contents vii 12 Dynamic Programming 12.1 12.2 12.3 12.4 12.5 12.6 12.7 323 Matrix Chain-Products The General Technique Telescope Scheduling Game Strategies The Longest Common Subsequence Problem The 0-1 Knapsack Problem Exercises 13 Graphs and Traversals 13.1 13.2 13.3 13.4 13.5 13.6 353 Graph Terminology and Representations Depth-First Search Breadth-First Search Directed Graphs Biconnected Components Exercises Part IV: 397 Single-Source Shortest Paths Dijkstra’s Algorithm The Bellman-Ford Algorithm Shortest Paths in Directed Acyclic Graphs All-Pairs Shortest Paths Exercises 15 Minimum Spanning Trees 15.1 15.2 15.3 15.4 15.5 CuuDuongThanCong.com Flows and Cuts Maximum Flow Algorithms Maximum Bipartite Matching Baseball Elimination Minimum-Cost Flow Exercises 399 400 407 410 412 418 423 Properties of Minimum Spanning Trees Kruskal’s Algorithm The Prim-Jarn´ık Algorithm ˚ Baruvka’s Algorithm Exercises 16 Network Flow and Matching 16.1 16.2 16.3 16.4 16.5 16.6 355 365 370 373 386 392 Graph Algorithms 14 Shortest Paths 14.1 14.2 14.3 14.4 14.5 14.6 325 329 331 334 339 343 346 425 428 433 436 439 443 445 452 458 460 462 469 Contents viii Part V: Computational Intractability 17 NP-Completeness 17.1 17.2 17.3 17.4 17.5 17.6 17.7 473 P and NP NP-Completeness CNF-SAT and 3SAT VERTEX-COVER, CLIQUE, and SET-COVER SUBSET-SUM and KNAPSACK HAMILTONIAN-CYCLE and TSP Exercises The Metric Traveling Salesperson Problem Approximations for Covering Problems Polynomial-Time Approximation Schemes Backtracking and Branch-and-Bound Exercises 18 Approximation Algorithms 18.1 18.2 18.3 18.4 18.5 Part VI: 507 529 Generating Random Permutations Stable Marriages and Coupon Collecting Minimum Cuts Finding Prime Numbers Chernoff Bounds Skip Lists Exercises 20 B-Trees and External Memory 20.1 20.2 20.3 20.4 20.5 External Memory (2,4) Trees and B-Trees External-Memory Sorting Online Caching Algorithms Exercises CuuDuongThanCong.com Range Trees Priority Search Trees Quadtrees and k-d Trees Exercises 531 534 539 546 551 557 563 569 21 Multidimensional Searching 21.1 21.2 21.3 21.4 511 515 518 521 525 Additional Topics 19 Randomized Algorithms 19.1 19.2 19.3 19.4 19.5 19.6 19.7 476 483 489 492 496 499 502 571 574 590 593 600 603 605 609 614 618 Bibliography 771 [155] N Megiddo, “Linear programming in linear time when the dimension is fixed,” J ACM, vol 31, pp 114–127, 1984 [156] K Mehlhorn, “A best possible bound for the weighted path length of binary search trees,” SIAM Journal on Computing, vol 6, no 2, pp 235–239, 1977 [157] K Mehlhorn, Data Structures and Algorithms 1: Sorting and Searching, vol of EATCS Monographs on Theoretical Computer Science Heidelberg, Germany: Springer-Verlag, 1984 [158] K Mehlhorn, Data Structures and Algorithms 2: Graph Algorithms and NP-Completeness, vol of EATCS Monographs on Theoretical Computer Science Heidelberg, Germany: Springer-Verlag, 1984 [159] K Mehlhorn, Data Structures and Algorithms 3: Multi-dimensional Searching and Computational Geometry, vol of EATCS Monographs on Theoretical Computer Science Heidelberg, Germany: Springer-Verlag, 1984 [160] K Mehlhorn and A Tsakalidis, “Data structures,” in Handbook of Theoretical Computer Science (J van Leeuwen, ed.), vol A Algorithms and Complexity, pp 301–341, Amsterdam: Elsevier, 1990 [161] D R Morrison, “PATRICIA—practical algorithm to retrieve information coded in alphanumeric,” Journal of the ACM, vol 15, no 4, pp 514–534, 1968 [162] R Motwani and P Raghavan, Randomized Algorithms New York: Cambridge University Press, 1995 [163] D R Musser and A Saini, STL Tutorial and Reference Guide: C++ Programming with the Standard Template Library Reading, MA: Addison-Wesley, 1996 [164] R Neapolitan and K Naimipour, Foundations of Algorithms Using C++ Pseudocode Boston: Jones and Bartlett Publishers, 1998 [165] J O’Rourke, Computational Geometry in C Cambridge University Press, 1994 [166] J Pach, ed., New Trends in Discrete and Computational Geometry, vol 10 of Algorithms and Combinatorics Springer-Verlag, 1993 [167] R Pagh and F Rodler, “Cuckoo hashing,” Journal of Algorithms, vol 52, pp 122–144, 2004 [168] T Papadakis, J I Munro, and P V Poblete, “Average search and update costs in skip lists,” BIT, vol 32, pp 316–332, 1992 [169] C H Papadimitriou and K Steiglitz, “Some complexity results for the traveling salesman problem,” in Proc 8th Annu ACM Sympos Theory Comput., pp 1–9, 1976 [170] C H Papadimitriou and K Steiglitz, Combinatorial Optimization: Algorithms and Complexity Englewood Cliffs, NJ: Prentice Hall, 1982 [171] M Patrascu and M Thorup, “The power of simple tabulation hashing,” in 43rd ACM Symp on Theory of Computing (STOC), pp 1–10, 2011 [172] P V Poblete, J I Munro, and T Papadakis, “The binomial transform and its application to the analysis of skip lists,” in Proceedings of the European Symposium on Algorithms (ESA), pp 554–569, 1995 [173] F P Preparata and M I Shamos, Computational Geometry: An Introduction Springer-Verlag, 3rd ed., Oct 1990 [174] R C Prim, “Shortest connection networks and some generalizations,” Bell Syst Tech J., vol 36, pp 1389–1401, 1957 [175] W Pugh, “Skip lists: a probabilistic alternative to balanced trees,” Communications of the ACM, vol 33, no 6, pp 668–676, 1990 [176] M Raab and A Steger, ““balls into bins” – a simple and tight analysis,” in Randomization and Approximation Techniques in Computer Science (RANDOM) (M Luby, J Rolim, and M Serna, eds.), vol 1518 of LNCS, pp 159–170, Springer, 1998 [177] M O Rabin, “A probabilistic algorithm for testing primality,” Journal of Number Theory, vol 12, 1980 [178] R L Rivest, A Shamir, and L Adleman, “A method for obtaining digital signatures and public-key cryptosystems,” Communications of the ACM, vol 21, no 2, pp 120–126, 1978 [179] D J Rosenkrantz, R E Stearns, and P M Lewis, “An analysis of several heuristics for the traveling salesman problem,” SIAM J on Computing, vol 6, pp 563–581, 1977 CuuDuongThanCong.com Bibliography 772 [180] H Sackrowitz, “Refining the point(s)-after-touchdown decision,” Chance, vol 13, no 3, pp 29–34, 2000 [181] H Samet, Applications of Spatial Data Structures: Computer Graphics, Image Processing, and GIS Reading, MA: Addison-Wesley, 1990 [182] H Samet, The Design and Analysis of Spatial Data Structures Reading, MA: AddisonWesley, 1990 [183] D Sankoff and J B Kruskal, Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison Addison-Wesley, 1983 [184] J E Savage, Models of Computation: Exploring the Power of Computing Addison-Wesley, 1998 [185] R Schaffer and R Sedgewick, “The analysis of heapsort,” Journal of Algorithms, vol 15, no 1, pp 76–100, 1993 [186] A Schăonhage and V Strassen, Schnelle multiplikation grosser zahlen,” Computing, vol 7, pp 281–292, 1971 [187] B L Schwartz, “Possible winners in partially completed tournaments,” SIAM Review, vol 8, no 3, pp 302–308, 1966 [188] R Sedgewick, Algorithms Reading, MA: Addison-Wesley, 1st ed., 1983 [189] R Sedgewick, Algorithms in C++ Reading, MA: Addison-Wesley, 1992 [190] R Sedgewick and P Flajolet, An Introduction to the Analysis of Algorithms Reading, MA: Addison-Wesley, 1996 [191] R Seidel and C Aragon, “Randomized search trees,” Algorithmica, vol 16, no 4-5, pp 464– 497, 1996 [192] R Seidel and M Sharir, “Top-down analysis of path compression,” SIAM Journal on Computing, vol 34, no 3, pp 515–525, 2005 [193] S Sen and R E Tarjan, “Deletion without rebalancing in balanced binary trees,” in Proc ACM-SIAM Symposium on Discrete Algorithms (SODA), pp 1490–1499, 2010 [194] A Shamir, “How to share a secret,” Comm of the ACM, vol 22(11), pp 612–613, 1979 [195] M Sipser, Introduction to the Theory of Computation PWS Publishing Co., 1997 [196] D D Sleator and R E Tarjan, “Amortized efficiency of list update and paging rules,” Comm ACM, vol 28, pp 202–208, 1985 [197] D D Sleator and R E Tarjan, “Self-adjusting binary search trees,” J ACM, vol 32, no 3, pp 652–686, 1985 [198] D K Smith, “Dynamic programming and board games: A survey,” European Journal of Operational Research, vol 176, no 3, pp 1299–1318, 2007 [199] D A Spielman and S.-H Teng, “Smoothed analysis of algorithms: Why the simplex algorithm usually takes polynomial time,” J ACM, vol 51, pp 385–463, May 2004 [200] G A Stephen, String Searching Algorithms World Scientific Press, 1994 [201] H S Stern, “American football,” in Statistics in Sport (J Bennett, ed.), Arnold, 1998 [202] V Strassen, “Gaussian elimination is not optimal,” Numerische Mathematik, vol 13, pp 354– 356, 1969 [203] R Tamassia, ed., Handbook of Graph Drawing and Visualization Chapman and Hall/CRC, 2013 [204] R Tarjan and U Vishkin, “An efficient parallel biconnectivity algorithm,” SIAM J Comput., vol 14, pp 862–874, 1985 [205] R E Tarjan, “Depth first search and linear graph algorithms,” SIAM Journal on Computing, vol 1, no 2, pp 146–160, 1972 [206] R E Tarjan, “A class of algorithms which require nonlinear time to maintain disjoint sets,” J Comput System Sci., vol 18, pp 110–127, 1979 [207] R E Tarjan, Data Structures and Network Algorithms, vol 44 of CBMS-NSF Regional Conference Series in Applied Mathematics SIAM, 1983 [208] R E Tarjan, “Amortized computational complexity,” SIAM J Algebraic Discrete Methods, vol 6, no 2, pp 306–318, 1985 CuuDuongThanCong.com Bibliography 773 [209] R E Tarjan and J van Leeuwen, “Worst-case analysis of set union algorithms,” J ACM, vol 31, pp 245–281, Mar 1984 [210] J van Leeuwen, “Graph algorithms,” in Handbook of Theoretical Computer Science (J van Leeuwen, ed.), vol A Algorithms and Complexity, pp 525–632, Amsterdam: Elsevier, 1990 [211] J S Vitter, Algorithms and Data Structures for External Memory Foundations and Trends in Theoretical Computer Science, Hanover, MA: Now Publishers, 2008 [212] J S Vitter and W C Chen, Design and Analysis of Coalesced Hashing New York: Oxford University Press, 1987 [213] J Vuillemin, “A unifying look at data structures,” Comm ACM, vol 23, pp 229–239, 1980 [214] S Warshall, “A theorem on boolean matrices,” Journal of the ACM, vol 9, no 1, pp 11–12, 1962 [215] K Wayne, “A new property and a faster algorithm for baseball elimination,” SIAM Journal on Discrete Mathematics, vol 14, no 2, pp 223–229, 2001 [216] J W J Williams, “Algorithm 232: Heapsort,” Comm ACM, vol 7, no 6, pp 347–348, 1964 [217] D Wood, Data Structures, Algorithms, and Performance Reading, MA: Addison-Wesley, 1993 [218] F F Yao, “Computational geometry,” in Algorithms in Complexity (R A Earnshaw and B Wyvill, eds.), pp 345–490, Amsterdam: Elsevier, 1990 [219] C K Yap, Fundamental Problems in Algorithmic Algebra Oxford University Press, 1999 CuuDuongThanCong.com Index δ-approximation, 510 2SAT, 491, 503 3SAT, 491, 492 (a, b) tree, 586–588 depth property, 586 size property, 586 accepting a string, 477 access control lists, 700 accounting method, 36–37 accumulator, 30, 33 Achter, 710 Ackermann function, 231, 232, 432 acyclic, 373 additive inverse, 691 Adel’son-Vel’skii, 154 adjacency list, 361 adjacency matrix, 361, 363 adjacent, 356 Adleman, 703 Agarwal, 239, 622 Aggarwal, 602 Aho, 88, 154, 506, 602, 684, 730 Ahuja, 396, 422, 472 algorithm, algorithm analysis, 6–33 average case, worst case, alphabet, 654 Alstrup, 239 amortization, 34–41, 73, 85, 145–148, 151, 221–235, 516–517, 662 accounting method, 36–37 potential function, 37–38 ancestor, 68, 372 anchor point, 632, 634–636 antisymmetric property, 157 approximation algorithms, 509–517 Aragon, 568, 622 Arge, 602 Arkin, 568 Arnold, 88 art gallery guarding, 300 Arya, 622 asymmetric relation, 356 asymptotic notation, 11–33 big-Oh, 11–14 big-Omega, 14 big-Theta, 14 774 CuuDuongThanCong.com little-oh, 16 little-omega, 16 augmented trees, 104 augmenting cycle, 462 augmenting path, 450 Aurenhammer, 650 AVL tree, 120–124, 644 height-balance property, 120 Baase, 730 back edge, 367, 375, 376, 393 backtracking, 365, 521–523, 528 backward induction, 338 Baeza-Yates, 114, 154, 264, 282, 684 balanced binary search tree, 116–148 balanced search tree, 577 Bar˚uvka, 442 Bar˚uvka’s algorithm, 436–438 baseball, 352 baseball elimination, 460–461 basic solution, 740 basic variables, 739 Baumert, 528 Bayer, 154, 602 Bellman, 352, 422 Bellman-Ford algorithm, 407–409 Bentley, 114, 186, 322, 650 Berg, 650 Bertrand’s Postulate, 213 BFS, see breadth-first search BFS tree, 372 biconnected, 386 biconnected component, 386 big integers, 313 big-Oh notation, 11–14 big-Omega notation, 14 big-Theta notation, 14 bin packing, 153 binary encoding, 476 binary search, 91–93 binary search tree, 90, 94–100 insertion, 97 removal, 98–99 rotation, 119 trinode restructure, 119 binary space partitioning, 90 binary tree, 69, 76–82, 244 complete, 163 left child, 69 Index 775 level, 76 linked structure, 82 proper, 69 right child, 69 binomial coefficients, 347 binomial distribution, 552 binomial expansion, 762 biodiversity, 534 bipartite graph, 458 birthday paradox, 217, 567 blocking, 572 Bloom filters, 567 Booch, 88 Boolean circuit, 481 bootstrapping, 576 Borodin, 602 bottleneck, 457 bottleneck TSP, 526 boundary node, 102 bounding box, 320, 621 Boyer, 684 branch-and-bound, 524–526, 528 Brassard, 50, 528 Bratley, 528 breadth-first search, 370–372, 376 Brigham, 730 brute force, 654 B-tree, 588–589 bubble-sort, 183 bucket-sort, 267–268 Budd, 88 buffer, 52 butterfly network, 725 c-incremental, 503 cache, 570 cache line, 572 caching algorithms, 594–599 call-by-value, 56 Canny, 568 capacity rule, 445 Carlsson, 186 Carmichael numbers, 547 Cartesian tree, 622 Catalan number, 326 certificate, 479 character-jump heuristic, 656 Chen, 218 Chernoff bound, 108, 109, 260, 551–553 child, 68 Chomsky normal form, 348 Christofides approximation algorithm, 513 Chv´atal, 528 CuuDuongThanCong.com CIRCUIT-SAT, 481, 484–486, 489 Clarkson, 282 clauses, 489 clearable table, 34 CLIQUE, 494, 502 clique, 494 clockwise, 627 closed form, 10 closest pairs, 642–645 closure, 415 CNF, see conjunctive normal form CNF-SAT, 489–491, 503, 506, 522, 523 coalesced hashing, 218 coefficient form, 713 coefficients, 734 coins-in-a-line game, 334 Cole, 684 collinear, 627 collision, 193 collision resolution, 198–214 collision-resistant, 702 Comer, 602 comparator, 179 compare-exchange, 261 complement, 478 complete binary tree, 163 complexity class, 478 component design, 487 composite, 546, 687 compositeness witness function, 547 compressed quadtree, 620 computational geometry, 624–650 closest pairs, 642–645 convex hull, 630–636 degeneracies, 628 orientation, 627–629 plane sweep, 640–645 proximity, 642 segment intersection, 638–641 conditional probability, 27 confidence parameter, 548 confidentiality, 686 congruence, 546 conjunctive normal form, 489, 491 connected components, 220, 221, 359, 369, 372 conservation rule, 445 constraints, 734 container, 88 contraction, 540 contradiction, 23 contrapositive, 23 convex hull, 630–636, 650 Index 776 gift wrapping, 632–634 Graham scan algorithm, 635–636 convexity, 629, 630, 736 convolution, 713 Convolution Theorem, 720 Cook, 506 Cook-Levin Theorem, 484 Cooley, 730 coordinates, 605 Cormen, 239, 322, 396, 506 cost, 462 counterclockwise, 627 coupon collector problem, 534–535 Crochemore, 684 cross edge, 372, 375, 376 cryptography, 699–707 El Gamal cryptosystem, 706–707 RSA cryptosystem, 703–705 cuckoo hashing, 206–211 currency arbitrage, 422 cursor, 64 cut, 447, 539 cyber-dollar, 36 cycle, 358 directed, 358 cyptograph public-key, 701–707 DAG, see directed acyclic graph data integrity, 686, 699 data structure, secondary, 576 data-oblivious, 261 decision problem, 477 decision tree, 257 decryption, 699 degeneracy, 744 degree, 356, 713 Delaunay triangulation, 650 DeMorgan’s law, 23 depth, 72–73 depth-first search, 365–369 deque, 85 descendant, 68, 372 destination, 356 DFS, see depth-first search DFS tree, 367 DFT, see Discrete Fourier Transform Di Battista, 88 diameter, 87, 647 dictionary, 189 update operations, 560 digital signature, 699 CuuDuongThanCong.com El Gamal, 707 RSA, 704 digraph, 373 Dijkstra, 422 Dijkstra’s algorithm, 400–406 Dillencourt, 239 directed acyclic graph, 382–384 directed cycle, 373 directed DFS tree, 375 discovery edge, 367, 372, 375, 376 Discrete Fourier Transform, 717–720 discrete logarithm, 706 disjunctive normal form, 490 distance, 399 divide-and-conquer, 243–245, 250–251, 305– 316 DNF, see disjunctive normal form DNF-DISSAT, 502 DNF-SAT, 502 d-node, 574 Doom, 90 double hashing, 204 double-entry accounting, 216 down-heap bubbling, 170, 176 dual LP, 746 Durstenfeld, 568 dynamic programming, 324–345, 377 dynamic time warping, 351 Easley, 239, 396, 422, 472 Edelsbrunner, 622, 650 edge, 355, 629 destination, 356 end vertices, 356 incident, 356 multiple, 357 origin, 356 outgoing, 356 parallel, 357 self-loop, 357 edge capacity, 445 edit distance, 347 Edmonds, 302, 472 Edmonds-Karp algorithm, 455–457 El-Yaniv, 602 Elliott, 730 El Gamal cryptosystem, 706–707 Emiris, 730 encryption schemes, 700 end vertices, 356 endpoints, 356 equivalence class, 387 equivalence relation, 387 Index 777 Euclid’s algorithm, 689–690 extended, 697–698 Euler tour, 392, 395 Euler tour traversal, 79, 88 Euler’s Theorem, 693 Even, 396, 422, 472 event, 26 exchange argument, 288 expected value, 27 exponent, 21 extended Euclid’s algorithm, 697 external memory, 571–592, 602 external-memory algorithm, 571–592 external-memory sorting, 590–592 factorial, 762 failure function, 660 Fast Fourier Transform, 713–726, 730 feasible region, 736 feasible solution, 736 Fermat’s Little Theorem, 546 FFT, see Fast Fourier Transform Fibonacci progression, 763 Fibonacci sequence, 150 FIFO, 57, 592 first-in first-out, 57 Fisher, 568 Fisher-Yates algorithm, 533 Flajolet, 50 flow, see network flow flow network, 445 Floyd, 186 Floyd-Warshall algorithm, 377, 396 Ford, 422 Ford-Fulkerson algorithm, 449–457 forest, 359 forward edge, 375 frame, 55 Frederickson, 239 free variables, 739 free-space optical communication, 441 Fulkerson, 472 fully polynomial-time approximation scheme, 518 Fundamental Theorem of Arithmetic, 687 fusion, 582, 587, 589 Gabow, 239 Gale, 568 game strategies, 334–338 garbage collection, 380–381 mark-sweep, 380 Garey, 506 CuuDuongThanCong.com Gauss, 20 Gavril, 528 GCD, see greatest common divisor generator, 694 geometric random variable, 534 geometric sum, 763 Gibbons, 396, 422, 472 gift wrapping, 632–634 Godbole, 352 Golberg, 88 golden ratio, 150, 763 Golomb, 528 Gonnet, 114, 154, 186, 264, 282 Goodman, 650 Goodrich, 602, 650 googol, 15 Gosling, 88 Graham, 50, 442, 650 Graham scan algorithm, 635–636 grammar, 348 graph, 354–396, 398–422, 424–442 acyclic, 373 bipartite, 458 breadth-first search, 370–372, 374–377 connected, 359, 372 data structures, 361–363 adjacency list, 361–362 adjacency matrix, 363 dense, 378 depth-first search, 365–369, 374–377 digraph, 373 directed, 355, 356, 373–384 acyclic, 382–384 strongly connected, 373 mixed, 356 reachability, 373–374, 377–378 shortest paths, 377–378 simple, 357 traversal, 365–372 undirected, 355, 356 weighted, 398–422, 424–442 greatest common divisor, 688–690 Euclid’s algorithm, 689–690 greedy method, 284–297, 399, 400 greedy-choice, 285, 297 group, 693 guess-and-test, 308–309 Guibas, 154 Guttag, 88 Haken, 322 HAMILTONIAN-CYCLE, 499, 501, 506, 526 harmonic number, 108, 112, 517, 535, 764 Index 778 hash function, 192, 204, 212, 213 2-universal, 212 one-way, 702 hash table, 188, 190–211 chaining, 198 clustering, 203 collision resolution, 198–214 double hashing, 204 linear probing, 200 open addressing, 204 quadratic probing, 204 secondary clustering, 204 universal hashing, 212–214 header, 64 heap, 163–178 heap-order property, 163 heap-sort, 174–178 height, 72–73 height-balance property, 120–123 Hell, 442 Hex game, 238 hidden-line elimination, 648 high probability, 532 Hilbert, 231 Hinrichs, 650 Hirschberg, 352 Hoare, 264 Hochbaum, 528 Hopcroft, 88, 154, 239, 396, 506, 602, 730 Horner’s method, 46 Horner’s rule, 713 Hu, 352 Huang, 264 Huffman, 302 Huffman coding, 292–297 Ibarra, 528 identity matrix, 718 in place, 160, 162 in-degree, 356 in-place, 254, 381 incidence matrix, 496 incident, 356 incoming edges, 356 independent, 26, 28 independent set, 348, 395 INDEPENDENT-SET, 504 index, 60, see discrete logarithm induction, 23–24 inorder traversal, 95, 99, 118 input size, 476 insertion-sort, 161 inside node, 102 CuuDuongThanCong.com integer multiplication, 313–314 internal memory, 570 intersection, 626 inverse shuffle, 725 inversion, 183, 260 inverted file, 242, 280 items, 189 iterative substitution, 306 J´aJ´a, 88 Jarn´ık, 442 Johnson, 506, 528 Johnston, 352 Jones, 396 Josephus problem, 87, 114 Kao, 264 Kaplan, 239 Karatsuba, 322 Karger, 442 Karp, 88, 472, 506 Karp-Rabin algorithm, 664, 666, 667 k-d tree, 616–617 key, 157, 189, 190, 574 key transfer, 701 Kim, 528 Klein, 442, 528 Kleinberg, 239, 396, 422, 472 KNAPSACK, 498, 504, 518–520, 525, 526, 528 knapsack problem, 286–288, 343–345 Knuth, 50, 88, 114, 154, 186, 264, 282, 396, 602, 684 Kosaraju, 396 Koutsoupias, 602 Kruskal, 352, 442 Kruskal’s algorithm, 428–432 Kung, 322 L’Hˆopital’s Rule, 764 Landis, 154 Langston, 264 language, 477 Las Vegas algorithm, 550 last node, 163 last-in first-out, 53 LCS, see longest common subsequence leaves, 68 Lecroq, 684 Lee, 650 left child, 69 left subtree, 69 left turn, 627 Index 779 Leiserson, 396, 506 level, 76, 370 level numbering, 165 level order traversal, 87 Levin, 506 Levisse, 114 Lewis, 528 lexicographic, 242 lexicographical, 268 lexicon matching, 664 LIFO, 53 line, 625 line breaking, 301 linear exponential, 764 linear function, 734 linear probing, 200 linear program, 734 linear programming, 732–752 linearity of expectation, 27, 272 link components, 389 link relation, 387 linked list, 63, 64 doubly linked, 64–66 linked structure, 82 Liskov, 88 list, 60–66 literals, 489 little-oh notation, 16 little-omega notation, 16 live objects, 380 load factor, 199, 205 local replacement, 487 locality-of-reference, 572 locator, 179–181 logarithm, 21, 761 natural, 761 longest common subsequence, 339–342 longest common substring, 348 looking-glass heuristic, 656 lookup table, 191 loop invariant, 25 Lov´asz, 528 LRU, 592 Luccio, 322 machine precision, 263, 300 machine scheduling, 300 Magnanti, 396, 422, 472 making change, 349 map, 189–211 hash table, 190–211 skip list, 557–562 mark-sweep algorithm, 380 CuuDuongThanCong.com Marker strategy, 597 Markov’s inequality, 551 master theorem, 310–312 matching, 458 matrix chain-product, 325–328 matrix closure, 415–417 matrix multiplication, 315–316 matroid theory, 302 Max-Flow, Min-Cut Theorem, 451 maxima set, 304, 317–318 maximum bipartite matching, 458–459 maximum flow, see network flow maximum spanning tree, 441 maximum subarray problem, 29 maximum subarray sum, 29 maze, 223 McCreight, 602, 622, 684 McDiarmid, 186 McGeoch, 50 median, 270 Megiddo, 282 Mehlhorn, 114, 154, 396, 422, 472, 602, 622, 650 memoization, 333, 336, 338 memory hierarchy, 570 memory management, 380, 571–573, 594–599 merge-sort, 243–249 multi-way, 590–592 tree, 244 mergeable heap, 151 method stack, 55–56 metric, 511 METRIC-TSP, 511–514 metrology, 281, 648 Milgram, 566 Miller, 568 minimum cut, 448, 539 minimum spanning tree, 425–438 Bar˚uvka’s algorithm, 436–438 Kruskal’s algorithm, 428–432 Prim-Jarnik algorithm, 433–435 minimum-cost flow, 462–468 mode, 260, 280 modular arithmetic, 58, 213, 546, 691–694, 762 modular exponentiation, 695–696 modular multiplicative inverse, 697 modulus, 546 Monte Carlo algorithm, 550 Moore, 684 Morris, 684 Morrison, 684 Index 780 Motwani, 264, 568, 602 MST, see minimum spanning tree multi-objective optimization, 317 multi-way search tree, 574 multi-way tree, 574–577 multimap, 190, 216 multiplicative group, 693 multiplicative inverse, 691, 697 Munro, 186 mutually independent, 26 Naimipour, 528 natural logarithm, 761 Neapolitan, 528 nearest-neighbor query, 116 negative binomial distribution, 554 network flow, 444–472 augmenting cycle, 462 augmenting path, 450 backward edge, 447 bottleneck, 457 capacity rule, 445 conservation rule, 445 cut, 447–449 cut capacity, 448 edge capacity, 445 Edmonds-Karp algorithm, 455–457 flow across a cut, 448 flow network, 445–447 flow value, 446 Ford-Fulkerson algorithm, 449–457 forward edge, 447 Max-Flow, Min-Cut Theorem, 451 maximum flow problem, 446, 449 minimum cut, 448 minimum-cost flow, 462–468 residual capacity, 449 residual distance, 455 residual graph, 454 node, 63, 68, 70 ancestor, 68 balanced, 121 boundary, 102 child, 68 descendant, 68 external, 68 inside, 102 internal, 68 outside, 102 parent, 68 redundant, 673 root, 68 sibling, 68 CuuDuongThanCong.com size, 145 unbalanced, 121 nonnegativity constraints, 734 nontree edge, 375, 376 NP, 478, 479 NP-completeness, 474–506 NP-hard, 483 null string, 653 number theory, 686–698 O’Rourke, 650 object-oriented design, 88 objective function, 285, 734 octree, 620 odd-even merge-sort, 261 office party problem, 350 offline-min problem, 237 Ofman, 322 one-sided error, 550 one-time pad, 709 one-way hash function, 702, 710 online algorithm, 593–599 open addressing, 200, 204 optimal binary search tree, 348 optimization problem, 329, 477 order statistic, 270 orientation, 626–628 origin, 356 Orlin, 396, 422, 472 orthogonal segments, 638 Ottmann, 650 out-degree, 356 outgoing edge, 356 output sensitive, 634, 638 outside node, 102 overflow, 579, 589 P, 478 Pach, 650 palindrome, 350 Pan, 730 Papadimitriou, 302, 528, 602 parent, 68 PARTITION, 504 partition, 221 partition tree, 614 password, 699 Patashnik, 50 path, 358 directed, 358 length, 399 simple, 358 weight, 399 Index 781 path compression, 229 path length, 87 pattern matching, 653–663 Boyer-Moore algorithm, 656–658 brute force, 654–656 Knuth-Morris-Pratt algorithm, 660–663 percolation theory, 224, 237 perfect matching, 513 pivot, 740 plane sweep, 640–645 point, 625 polygon, 625, 629 convex, 629 edges, 629 simple, 629 vertices, 629 polynomial, 46 polynomial-time approximation scheme, 518 polynomial-time reducible, 483 position, 63, 70 potential function, 37–38 Pratt, 684 prefix, 653 prefix code, 292 prefix sums, 31 Preparata, 322, 650 Prim, 442 Prim-Jarnik algorithm, 433–435 primality testing, 547–549 prime, 546, 687 prime decomposition, 687 primitive operations, 7–10, 36 primitive root, 694 primitive root of unity, 715 printer spooler, 59 priority queue, 157–181 priority range tree, 613–614 priority search tree, 609–612 probabilistic packet marking, 566 probability, 26–28 probability space, 26 producer-consumer model, 52 program counter, 56 proximity, 642 prune-and-search, 270–278 pseudo-blocks, 592 pseudo-polynomial-time, 345 pseudo-random-number generators, 530 pseudocode, 5–6 PTAS, see polynomial-time approximation scheme public-key cryptography, 701–707 public-key cryptosystem, 701 Pugh, 568 CuuDuongThanCong.com quadratic probing, 204 quadtree, 614–615 queue, 52, 57–59 array implementation, 57–59 quick-sort, 250–256 tree, 251 Raab, 568 Rabin, 568 Rabin-Miller algorithm, 548 radix-sort, 268–269 Raghavan, 264, 568, 602 Ramachandran, 88 random binary search tree, 107 random permutation, 531–534 random variable, 27 random-access machine, randomized algorithm, 530 randomized quick-select, 270 randomized quick-sort, 253 range query, 101 range searching one-dimensional, 101–103 three-sided, 609 range tree, 605–614 range-search query, 605 rank-balanced trees, 118 Rao, 730 reachability, 373 recurrence equation, 10, 248, 305–312 recursion, 10, 56 recursion tree, 307 red-black tree, 126–129, 644 Reed, 186 reflexive property, 157 rehashing, 205 relatively prime, 688, 691 relaxation, 400, 407 repeated squaring, 695 rescalable, 518 residual capacity, 449 residual distance, 455 residual graph, 454 residue, 691 restriction, 487 restructure trinode, 119 Ribeiro-Neto, 684 riffle shuffle, 262 right child, 69 right subtree, 69 right turn, 627 ring-loading problem, 527 Index 782 Rivest, 396, 506, 703 road network, 398 Robson, 88 rolling-hash function, 666 root, 68 root object, 380 Rosenkrantz, 528 rotation, 118, 119, 122 double, 119 single, 119 roundoff error, 263, 300 RSA cryptosystem, 703–705 running time, 2–4, 8–10 Sackrowitz, 352 Samet, 602, 622 sample space, 26 Sankoff, 352 SAT, 502 satisfying assignment, 481 Saxe, 322 scalability, scan forward, 558 Schaffer, 186 search engine, 679 secondary clustering, 204 Sedgewick, 50, 154, 186, 650 seed, 530 segment, 625 segment intersection, 638–641 Seidel, 239, 568, 622 selection, 270–278 selection-sort, 159 self-loop, 357 sentinel, 64, 190 separate chaining, 198 separation edge, 386 separation vertex, 386 set cover, 509 SET-COVER, 494, 495, 502, 515–517, 528 Shamir, 703 Shamir secret sharing, 728 Shamos, 650 Shapley, 568 Sharir, 239 Shing, 352 shortest path, 399–417 Bellman-Ford algorithm, 407–409 Dijkstra’s algorithm, 400–406 matrix multiplication, 413–417 sibling, 68 simplex method, 733, 739 sink, 444, 445 CuuDuongThanCong.com ski rental problem, 593 skip list, 557–562 analysis, 561–562 insertion, 559 levels, 557 removal, 560–561 searching, 558–559 towers, 557 update operations, 559–561 skyline problem, 322 slack form, 739 Sleator, 154, 602 Smith-Waterman algorithm, 350 social network, 220 sorting, 158, 243–258, 267–269 bucket-sort, 267–268 external-memory, 590–592 heap-sort, 174–178 in-place, 175, 254 insertion-sort, 161 lower bound, 257–258 merge-sort, 243–249 priority-queue, 158 quick-sort, 250–256 radix-sort, 268–269 selection-sort, 159 stable, 268 source, 444, 445 space usage, spanning forest, 369, 372 spanning subgraph, 359 spanning tree, 359, 365, 367, 368, 370, 372, 425 splay tree, 139–148 split, 579, 589 stable, 268 stable marriage problem, 536–538 stack, 53–56 array implementation, 53–54 standard form, 734 Stearns, 528 steganography, 682 Steger, 568 Steiglitz, 302, 528 Stephen, 684 Stirling’s Approximation, 762 STL, 88 Stooge-sort, 320 stop words, 671, 682 Strassen’s Algorithm, 315 string null, 653 pattern matching, see pattern matching Index 783 prefix, 653 suffix, 653 strongly collision-resistant, 702 strongly connected, 373 strongly NP-hard, 496 subgraph, 359 SUBGRAPH-ISOMORPHISM, 504 subproblem optimality, 326, 329 subproblem overlap, 329 subsequence, 302, 324, 339 SUBSET-SUM, 496–498, 502, 506 substring, 348, 653 subtree, 68 suffix, 653 summation, 19, 763 geometric, 19 symmetric encryption, 700–701 symmetric relation, 355 tabulation-based hashing, 196 tail estimate, 535 tail recursion, 256 Tamassia, 88, 710 Tarjan, 50, 88, 114, 154, 239, 396, 422, 442, 472, 602 task scheduling, 289–291 telescope scheduling, 331–333, 350 telescoping sum, 35, 763 text compression, 292–297 topological ordering, 382–384 total order, 157 totient function, 693 tower, 559 traceback, 566 trailer, 64 transfer, 582 transitive closure, 374, 376 transitive property, 157 traveling salesperson problem, 480, 501 treap, 622 tree, 68–88, 359 binary, see binary tree child node, 68 depth, 72–73 external node, 68 height, 72–73 internal node, 68 level, 76 linked structure, 83–84 multi-way, 574–577 multidimensional, 605–617 node, 68 CuuDuongThanCong.com ordered, 69 parent node, 68 root node, 68 tree edge, 375, 376 tree traversal, 74–75, 78–81 Euler tour, 80–81 generic, 80–81 inorder, 79 level order, 87 postorder, 75, 78 preorder, 74, 78 triangle inequality, 511 triangulation, 348 trie, 669–679 compressed, 673 standard, 669 trinode restructuring, 118, 122 Tsakalidis, 114, 154 TSP, 480, 501, 511, 512, 518, 524, 526, 528 Tukey, 730 two-dimensional dictionary, 605 two-dimensional pattern matching, 683 two-sided error, 550 (2, 4) tree, 578–584 depth property, 578 size property, 578 Ullman, 88, 154, 239, 506, 602, 730 unary encoding, 476 underflow, 582, 589 uniform hash, 664 union-by-size, 229 union-find, 221–235 universal hashing, 212–214 up-heap bubbling, 168 upper envelope, 321 van Leeuwen, 396, 422, 472 verification, 479 vertex, 355 degree, 356 in-degree, 356 out-degree, 356 vertex cover, 352, 471, 482 VERTEX-COVER, 482, 492–497, 499, 506, 515, 528 vertical ray-shooting, 648 virtual memory, 572 Vishkin, 88 Vitter, 218, 602 Voronoi diagram, 649 Vuillemin, 622 Index 784 Wang, 264 weak AVL tree, 130–135 web crawler, 339, 365 web spider, 365 weighted moving average, 730 Williams, 186 Wood, 88, 622 word wrapping, 349 Yannakakis, 528 Yao, 650 Yap, 730 Yates, 568 Young, 528 zig, 140, 146 zig-zag, 140, 146 zig-zig, 139, 146 CuuDuongThanCong.com WILEY END USER LICENSE AGREEMENT Go to www.wiley.com/go/eula to access Wiley’s ebook EULA CuuDuongThanCong.com ... (the maximum) 2 10 11 -2 -4 -6 sum = sum = -1 -8 A = [ -2 , -4 , 3, -1 , 5, 6, -7 , -2 , 4, -3 , 2] Figure 1.13: An instance of the maximum subarray problem In this case, the max- imum subarray is A[3... Merge-Sort and Quick-Sort 13 Graphs and Traversals 14 Shortest Paths 15 Minimum Spanning Trees 20 B-Trees and External-Memory 21 Multi-Dimensional Searching Example syllabus for an upper-division/graduate... Algorithm Analysis 14 Using the Big-Oh Notation It is considered poor taste, in general, to say “f (n) ≤ O(g(n)),” since the big-Oh already denotes the “less-than-or-equal-to” concept Likewise, although