CuuDuongThanCong.com bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 602 Essential Algorithms A Practical Approach to Computer Algorithms Rod Stephens ffi rs.indd CuuDuongThanCong.com 05:53:16:PM 07/10/2013 Page i Essential Algorithms: A Practical Approach to Computer Algorithms Published by John Wiley & Sons, Inc 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2013 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-61210-1 ISBN: 978-1-118-61276-7 (ebk) ISBN: 978-1-118-79729-7 (ebk) Manufactured in the United States of America 10 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600 Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley com/go/permissions Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose No warranty may be created or extended by sales or promotional materials The advice and strategies contained herein may not be suitable for every situation This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services If professional assistance is required, the services of a competent professional person should be sought Neither the publisher nor the author shall be liable for damages arising herefrom The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002 Wiley publishes in a variety of print and electronic formats and by print-on-demand Some material included with standard print versions of this book may not be included in e-books or in print-on-demand If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com For more information about Wiley products, visit www.wiley.com Library of Congress Control Number: 2013941603 Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc and/or its affiliates, in the United States and other countries, and may not be used without written permission All other trademarks are the property of their respective owners John Wiley & Sons, Inc is not associated with any product or vendor mentioned in this book ffi rs.indd 05:53:16:PM 07/10/2013 CuuDuongThanCong.com Page ii About the Author Rod Stephens started out as a mathematician, but while studying at MIT, he discovered how much fun algorithms are He took every algorithms course MIT offered and has been writing complex algorithms ever since During his career, Rod has worked on an eclectic assortment of applications in such fields as telephone switching, billing, repair dispatching, tax processing, wastewater treatment, concert ticket sales, cartography, and training for professional football players Rod is a Microsoft Visual Basic Most Valuable Professional (MVP) and has taught introductory programming at ITT Technical Institute He has written more than dozen books that have been translated into languages from all over the world He has also written more than 250 magazine articles covering C#, Visual Basic, Visual Basic for Applications, Delphi, and Java Rod’s popular VB Helper website (www.vb-helper.com) receives several million hits per month and contains tips, tricks, and example programs for Visual Basic programmers His C# Helper website (www.csharphelper.com) contains similar material for C# programmers You can contact Rod at RodStephens@vb-helper.com or RodStephens@csharphelper.com iii ffi rs.indd CuuDuongThanCong.com 05:53:16:PM 07/10/2013 Page iii Credits Executive Editor Robert Elliott Business Manager Amy Knies Project Editor Tom Dinse Production Manager Tim Tate Technical Editors David Coleman Jack Jianxiu Hao George Kocur Vice President and Executive Group Publisher Richard Swadley Production Editor Daniel Scribner Vice President and Executive Publisher Neil Edde Copy Editor Gayle Johnson Associate Publisher Jim Minatel Editorial Manager Mary Beth Wakefield Project Coordinator, Cover Katie Crocker Freelancer Editorial Manager Rosemarie Graham Proofreader Josh Chase, Word One Associate Director of Marketing David Mayhew Indexer Robert Swanson Marketing Manager Ashley Zurcher Cover Designer Ryan Sneed iv ffi rs.indd 05:53:16:PM 07/10/2013 CuuDuongThanCong.com Page iv Acknowledgments Thanks to Bob Elliott, Tom Dinse, Gayle Johnson, and Daniel Scribner for all of their hard work in making this book possible Thanks also to technical editors George Kocur, Dave Colman, and Jack Jianxiu Hao for helping ensure the information in this book is as accurate as possible (Any remaining mistakes are mine not theirs.) v ffi rs.indd CuuDuongThanCong.com 05:53:16:PM 07/10/2013 Page v Contents at a Glance Introduction Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter Chapter 10 Chapter 11 Chapter 12 Chapter 13 Chapter 14 Chapter 15 Chapter 16 Chapter 17 Chapter 18 Chapter 19 Appendix A Appendix B xv Algorithm Basics Numerical Algorithms Linked Lists Arrays Stacks and Queues Sorting Searching Hash Tables Recursion Trees Balanced Trees Decision Trees Basic Network Algorithms More Network Algorithms String Algorithms Cryptography Complexity Theory Distributed Algorithms Interview Puzzles Summary of Algorithmic Concepts Solutions to Exercises 25 55 83 111 131 163 169 185 227 277 297 325 355 377 397 419 435 465 477 487 Glossary 559 Index 573 vi ffi rs.indd 05:53:16:PM 07/10/2013 CuuDuongThanCong.com Page vi Contents Introduction Chapter Chapter xv Algorithm Basics Approach Algorithms and Data Structures Pseudocode Algorithm Features 3 Big O Notation Common Runtime Functions Visualizing Functions 11 17 Practical Considerations Summary Exercises 17 19 20 Numerical Algorithms Randomizing Data 25 25 Generating Random Values Randomizing Arrays Generating Nonuniform Distributions 25 31 33 Finding Greatest Common Divisors Performing Exponentiation Working with Prime Numbers 33 35 36 Finding Prime Factors Finding Primes Testing for Primality 37 39 40 Performing Numerical Integration 42 The Rectangle Rule The Trapezoid Rule 42 43 vii ftoc.indd CuuDuongThanCong.com 05:53:41:PM 07/10/2013 Page vii viii Contents Adaptive Quadrature Monte Carlo Integration Chapter Finding Zeros Summary Exercises 49 51 52 Linked Lists Basic Concepts Singly Linked Lists 55 55 56 Iterating Over the List Finding Cells Using Sentinels Adding Cells at the Beginning Adding Cells at the End Inserting Cells After Other Cells Deleting Cells Doubly Linked Lists Sorted Linked Lists Linked-List Algorithms 63 65 66 67 68 Linked List Selectionsort Multithreaded Linked Lists Linked Lists with Loops 69 70 71 72 74 75 76 78 80 Summary Exercises 81 81 Arrays Basic Concepts One-dimensional Arrays 83 83 86 Finding Items Finding Minimum, Maximum, and Average Inserting Items Removing Items 86 86 88 89 Nonzero Lower Bounds 89 Two Dimensions Higher Dimensions 90 91 Triangular Arrays Sparse Arrays ftoc.indd 57 57 58 59 60 61 62 Copying Lists Sorting with Insertionsort Marking Cells Using Hash Tables List Retracing List Reversal Tortoise and Hare Loops in Doubly Linked Lists Chapter 44 48 05:53:41:PM 07/10/2013 CuuDuongThanCong.com Page viii 94 97 588 Index ■ N–N MilkyWay@home, 439 minimal flow cut problem, 372–374, 376, 537 Minimal Flow Cut tool, 372, 537 minimal spanning trees, 329, 338–339, 350, 352, 483, 533 minimax strategy, 298–302, 322, 482 minimum, one-dimensional arrays, 86–88 minimum degree spanning tree, 430 minimum k-cut, 430 Misra/Chandy solution, 451, 462, 548 modeling accuracy, 478 See also rectangle rule; trapezoid rule modulus operator, 4, 34 Monte Carlo integration, 48, 52, 54, 478, 496 Moore See Boyer-Moore algorithm Moore, Gordon E., 435 Moore’s Law, 435, 462 MoveMemory, 18 multiheaded queue, 129, 510 MultiHeadedQueue example program, 510 multiple cores, 435, 440, 446, 462, 485 multiple CPUs, 155, 432, 438–439, 440, 449, 485 multiple linear congruential generators, 27 multiplication sparse matrices, 107, 109, 508–509 triangular arrays, 109, 505 two-dimensional matrices, 106 multiplicative inverse, 412, 413–414 MultiplicativeInverse program, 414, 418 multithreaded linked lists, 70–71, 82, 500 mutexes, 442–445, 447, 462, 548 N N, 15, 17, 478 N! See factorial function N log N, 15, 17, 419–420, 478 N2, 15, 17, 478 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 588 naughts and crosses See tic-tac-toe NDArray sample program, 94 neighbors, 325, 328, 329, 333, 334, 335 nested loops, 10, 347, 349–350 NET Framework Array.Sort, 18–19 BinarySearch method, 163 network algorithms, 325–376 exercises, 351–353, 375–376, 532–537 map coloring, 359–367 five-coloring, 363–367 four-coloring, 362–363, 367, 375, 484 hill-climbing strategy, 367 other algorithms, 367 three-coloring, 362, 363, 364, 367, 375, 484, 535, 537 two-coloring, 360–361, 375, 484, 537 summary of concepts, 483–484 topological sorting, 355–359, 375, 484 traversals, 331–339 breadth-first traversal, 334–335, 342, 350, 355, 483 depth-first traversal, 331–334, 335, 350, 355, 483, 520 NetworkMaker example program, 351, 532–533, 534, 535, 536, 537 networks augmenting paths, 369, 370, 373, 376, 484, 535 bipartite, 371, 376, 537 capacitated, 368–370, 376 connected components, 326, 327, 335, 336–337, 351, 359, 361, 362, 363, 364, 532 connectivity testing, 335–337, 350, 483 cycle, 326, 327, 328, 332 cycle detection, 359 directed, 326–327, 328, 329, 351, 532 distributed, 440 edges, 325, 327 finding paths, 339–350 all-pairs shortest paths, 345–350, 353, 483, 534 any path, 339–340 Index ■ N–N label-correcting shortest paths, 344–345, 352, 483, 533 label-setting shortest paths, 340– 343, 344, 350, 352, 483, 533 as graphs, 325 links backlinks, 369–370, 373 costs, 326, 327, 328, 329, 338, 340, 342, 344 defined, 326, 328 directed/undirected, 326, 328 edges, 325, 327 maximal flow, 368–370, 371, 373, 374, 376, 426, 484, 536 minimal flow cut problem, 372–374, 376, 537 node class, 328–332, 483 nodes adjacent, 326, 327, 328 neighbors, 325, 328, 329, 333, 334, 335 partial ordering, 356–357, 484 planar, 363, 367, 375, 430, 431 representations, 328–331, 483 residual capacity, 369–370, 373, 376, 448, 535 spanning trees defined, 337 described, 337–338 minimal, 329, 338–339, 350, 352, 483, 533 strongly connected, 326, 328 terminology, 325–328 trees compared to, 227, 322, 325, 332 undirected, 326, 327, 328, 329, 335, 337, 360, 545 weakly connected, 326, 328 work assignments, 370–371, 376, 426, 484, 536, 537 Newton-Cotes formulas, 42–44 Newton-Raphson method, 49–51, 54, 488, 496 NewtonsMethod sample program, 51 NEXPSPACE, 423 NEXPTIME, 423, 485 NextIndex1, NextIndex2 and, 115– 116, 128, 509 NFAs See nondeterministic finite automata node class, networks, 328–332, 483 node merges, 284, 285–286 node splits, 283–284, 291, 482 nodes See also networks; trees branches, 227–228, 230, 481 child, 228, 230 classes for, 234–236 defined networks, 325, 328 trees, 231 external, 228, 231 first common ancestor, 229, 231 height, 229, 231, 271 internal, 228, 231 leaf, 228, 231, 232, 481 level/depth, 229, 231 network adjacent, 326, 327, 328 defined, 325, 328 in-degree, 326, 327, 328, 375, 535 neighbors, 325, 328, 329, 333, 334, 335 out-degree, 326, 327, 328, 375, 535 reachable, 326, 327, 328, 335 parent, 228, 231 root, 228, 229, 231 sorted trees adding, 245–247 deleting, 248–250 finding, 247 3-node, 282 2-node, 282 vertices, 325, 430, 431 noise, atmospheric, 26 nondeterministic complexity classes, 423 nondeterministic computers, 421–422, 423, 424, 445, 485 nondeterministic finite automata (NFAs), 386–387 nondeterministic polynomial time See NP bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 589 589 590 Index ■ O–O nonindexed database searches, 448 nonrecursive Factorial algorithm, 217, 225, 518 nonrecursive Hilbert curve algorithm, 220, 221–222, 225, 518 NonrecursiveFactorial example program, 518 NonrecursiveFibonacci example program, 518 NonrecursiveFibonacci2 example program, 518 NonrecursiveHilbert example program, 518 nonuniform distributions million integers, 162, 512 PRNGs, 33 nonzero lower bounds, 89–91, 108, 479, 511 not_sorted, 136 NP defined, 423 NP-complete, 424, 425, 426, 427, 429–431 NP-hardness, 426–427 P equal NP question, 423, 431, 485 P is subset of NP, 423 quantum computing, 445 NPSPACE, 423 NTIME, 423, 485 null, 13 numeric quadrature See numerical integration numerical algorithms, 25–54 adaptive quadrature, 44–47, 52 data structures, 52 exercises, 52–54, 491–496 fast exponentiation, 35–36, 41, 53, 413, 478, 492–493 GCDs cryptographic routines, 33 Euclid’s algorithm, 33–35, 53, 413, 492 exercises, 53, 492, 494–495 GcdTimes example, 494 LCM, 53, 492 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 590 pseudocode example, 3–4 Monte Carlo integration, 48, 52, 54, 478, 496 prime factors Carmichael numbers and prime factors algorithm, 53, 494–495 described, 37–39 prime numbers, 36–41 composite numbers, 36, 41, 53 defined, 36 Fermat’s primality test, 6, 40, 52, 478 finding, 39–40 sieve of Eratosthenes, 39–40, 53, 422, 478, 494 testing for primality, 40–41 summary of concepts, 478 uses, 51 numerical integration, 42–48 rectangle rule, 42–43, 44, 478 trapezoid rule, 43–45, 478 O O(1), 9, 11 O(log N), 12, 14, 15 See also balanced trees O(N), 8, 9, 10, 11, 15 O(N log N), 15, 69, 88 See also sorting algorithms O(N2), 7, 10, 15, 18, 20 See also sorting algorithms O(N3), 11, 15, 350, 483, 489 O(sqrt(N)), 14, 38, 39 Θ(g(N)), 420 Θ(N log N), 420, 421 object references, 234, 481 obscurity, 397, 398, 399, 415, 540 octtrees, 266, 482 odd-cycle problem, 432, 543 (constant), 11, 17, 478 1,000 integers, 511 1,000 names, 512 100,000 integers with values between and billion, 161, 512 100,000 integers with values between and 1,000, 161, 512 Index ■ P–P 100,000 names, 162, 512 million floating-point values, 162, 512 million integers, 162, 512 one-dimensional arrays (linear arrays) exercises, 108, 501 finding items, 86 finding minimum, maximum, average, 86–88 graphical representation, 84 inserting items, 88–89 int[,] numbers = new int[10, 20];, 84 median of, 87–88, 108, 501–502 removing items, 89, 109, 502 sample variance, 108, 501 standard deviation, 108, 501 one-time pads, 404, 408, 418, 484, 541, 542 OneTimePad example program, 542 open addressing, 172–174, 182, 183, 480, 514 optimization problems See also decision trees bin packing, 318, 417, 429 complexity theory, 427–429, 482, 485 decision trees, 482 knapsack problem, 433, 546 partition problem, 306–310, 317, 323 ordered hashing, 179–181, 182, 515 ordered tree, 229, 231 OrderedDoubleHashing example program, 514 OrderedQuadraticHashing example program, 514 ordering, partial, 356–357, 484 out-degree, nodes, 326, 327, 328, 375, 535 P P See also NP defined, 422 P equal NP question, 423, 431, 485 P is subset of NP, 423 painting fences, 22, 185, 489 panic, interview puzzles, 468, 469, 473 parallel algorithms debugging, 485 embarrassingly parallel algorithms, 447–449, 485 multiple CPUs, 435 parallelism See also distributed algorithms bucketsort algorithm, 480 complexity theory, 432 data, 436, 440 distributed computing, 436, 438–440, 446, 462, 485 mergesort algorithm, 154, 155, 159, 480 multiple cores, 435, 440, 446, 462, 485 multiple CPUs, 155, 432, 438–439, 440, 449, 485 quantum computing, 435, 446, 485 quicksort algorithm, 152, 154, 159, 480 scarcity, 446 systolic arrays, 436–438, 446, 462, 485, 547 task, 440 types, 436, 485 parent node, 228, 231 parentheses, matching, 378–380, 381, 394 ParenthesisMatching example program, 537 Parkinson’s disease, Folding@home project, 439 parse trees, 380, 383, 387, 395, 396, 484, 539 partial ordering, 356–357, 484 partition problem, 305–317, 319, 320, 321, 323, 431, 433 PartitionProblem example program, 531–532 passing methods, preorder traversal, 240 paths, network See networks pattern matching, 381–387 P-boxes, 409 bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 591 591 592 Index ■ P–P perfect tree, 231 permutation boxes, 409 permutations defined, 209 with duplicates, 214–215 without duplicates, 215–216 petaflop, 439 phase king algorithm, 456–458 philosophers problem, dining, 445, 449–452, 462–463, 485, 548–549, 550 pictures balanced trees, 277 solving interview puzzles, 471 pill bottles, interview puzzles, 475, 558 placeholder See sentinels plaintext message, 398 planar network, 363, 367, 375, 430, 431 PlanetList example program, 500 planets, linked list, 70–71, 82, 500 playing cards See cards poker program, 52, 492 polynomial time See NP; P polynomials functions, 18 Newton-Cotes formulas, 42–44 runtimes, 15, 20, 483 polynomial-time reductions, 424, 425, 426, 431, 432, 433, 485, 544 popping items, 112, 113, 114, 115–116 postal services Chinese postman problem, 429 TSP, 320 postorder traversal, 242–243, 244, 272, 482, 520 precalculated values, fast exponentiation, 478 prefix trees, 266 See also tries preorder traversal, 238–240, 244, 271, 272, 331, 482, 520 Pretty Good Privacy program, 415 primary clustering, 175–177, 178 prime factors Carmichael numbers and prime factors algorithm, 53, 494–495 described, 37–39 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 592 prime numbers, 36–41 composite numbers, 36, 41, 53 defined, 36 Fermat’s primality test, 6, 40, 52, 478 finding, 39–40 sieve of Eratosthenes, 39–40, 53, 422, 478, 494 testing for primality, 40–41 priority inversion problem, 444 priority queues, 127, 129, 142, 161, 479, 510, 511 PriorityQueue example program, 511 PRNGs (pseudorandom number generators) biased, 29 CSPRNGs described, 28–29, 31 disadvantages, 29 unbreakable encryption scheme, 418, 542 dining philosophers problem, 450 fair, 29 linear congruential generators, 26–29 nonuniform distributions, 33 seed values, 26–28, 450 TRNGs, 25–26 probabilistic algorithm, 41, 51, 478 probabilistic computers, 445 probabilistic method, interview puzzles, 469 probe sequences, 173, 174, 175, 176, 177, 178, 179, 180, 181, 183, 513, 514, 515 probing linear, 174–176, 177, 178, 183, 480, 514, 515 pseudorandom, 178, 179, 181, 183, 515 quadratic, 176–178, 181, 183, 480, 514, 515 problem structure, interview puzzles, 469 problem-solving ability, 467, 473, 486 problem-solving tips, 470 proof-of-concept demonstrations, quantum computing, 445 proofs Index ■ Q–R four-coloring theorem, 363 inductive reasoning, tree theorems, 233, 481, 519–520 intuitive approach to algorithms, leaf and full nodes, binary trees, 232 random array, 32 pseudocode balanced trees, 277 described, 3–6, 477 problem, pseudorandom number generators See PRNGs pseudorandom probing, 178, 179, 181, 183, 515 PseudoRandomProbing example program, 514 PSPACE, 423 public grid projects, 439–440, 446, 462 public-key encryption, 412–413, 415, 416, 485 pushdown stack, 112 pushing items, 112, 113, 114, 115–116 puzzles See interview puzzles Q quadratic equation, 96 quadratic probing, 176–178, 181, 183, 480, 514, 515 QuadraticProbing example program, 177, 514 quadrature, adaptive, 44–47, 52 quadtrees, 260–266, 482, 523 quantum computing, 435, 446, 485 quantum-size transistors, 435, 445 qubits, 445 queens problem See eight queens problem QueueInsertionsort example program, 510 queues, 123–128 in arrays, 124–127 arrays compared to, 108 circular, 125–127 defined, 123, 479 deques, 127–128, 129, 479, 510 exercises, 128–129, 509–510 FIFO, 123, 128, 244, 334, 335, 479 insertionsort algorithm, 127, 129, 510 linked-list, 123–124 multiheaded, 129, 510 priority, 127, 129, 142, 161, 479, 510, 511 selectionsort algorithm, 129, 510 specialized, 127–128 summary of concepts, 479 uses, 111, 128, 479 QueueSelectionsort example program, 510 quicksort algorithm, 145–152 characteristics, 159 comparison, 159–160 countingsort compared to, 157 divide-and-conquer strategy, 153 exercises, 161, 511, 512 implementation in place, 149–152 with stacks, 149 mergesort compared to, 153–155 parallelism, 152, 154, 159, 480 picking dividing item, 148 randomization, 31, 479 runtime analysis, 146–148 sub O(N log N) algorithms, 156 upper and lower bounds, 420–421 using, 152 when to use, 19 worst-case performance O(N2), QuicksortQueue example program, 511 QuicksortStack example program, 511 R race conditions, 440–444, 446, 462, 548 radiation detector, 26 radio waves, static, 26 radioactive sample, 26 random number generators, true, 25–26 See also PRNGs random numbers, true, 26 random path selection, decision trees, 311–314 bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 593 593 594 Index ■ R–R random searches, 448 randomization, 25–33 arrays, 31–33 dining philosophers problem, 450 hash tables, 480 interview puzzles, 469 quicksort algorithm, 31, 479 summary, 478 Random.org, 26 RandomTrees program, 27–28 Raphson method See NewtonRaphson method ray tracing, 447 reachable node, 326, 327, 328, 335 receiver, 398 rectangle rule, 42–43, 44, 478 RectangleRule sample program, 42–43 rectangular arrays, 109, 503 recursion, 185–225 backtracking algorithms, 201–203 defined, 185 drawbacks, 216, 223 eight queens problem, 203–206, 208, 223, 224, 322, 518 exercises, 223–225, 515–518 factorial algorithm nonrecursive, 217, 225, 518 recursive, 223, 515 tail recursion, 216–217 factorial function (N!), 15, 16, 17, 18, 20, 186–188, 189, 478, 481 Fibonacci numbers, 188–189 graphical algorithms, 193–201 Hilbert curves, 196–197, 199, 200, 220–222, 224, 225, 516, 518 important features, 186 interpolation search, 168, 513 knight’s tour problem, 206–209, 216, 223, 225, 518 Koch curves, 193–195 Koch snowflakes, 195, 223–224, 516 linear search, 168, 512 memory, 187 permutations defined, 209 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 594 with duplicates, 214–215 without duplicates, 215–216 removal, 216 general algorithm for, 220–222, 481 storing intermediate values, 218– 219, 481 tail recursion removal, 216–218 Sierpinski carpet, 201, 224, 517–518 Sierpinski curves, 197–200, 224, 516, 517 Sierpinski gaskets, 200–201, 224, 517 summary of concepts, 480–481 Tower of Hanoi puzzle, 119–121, 189–193, 223, 515–516 RecursiveBinarySearch example program, 513 RecursiveInterpolationSearch example program, 513 RecursiveLinearSearch example program, 512 reductions described, 424–425, 485 polynomial-time reductions, 424, 425, 426, 431, 432, 433, 485, 544 regular expressions, 381–387, 394, 395, 484, 537, 538 relational databases, B-trees, 293 remodeling kitchen, 355–359, 375 removal See recursion removing items hash tables, 174 linear arrays, 89, 109, 502 reporting problem, 427–429, 433, 485, 546 ReportKnapsack, 546 residual capacities, 369–370, 373, 376 residual capacity networks, 369–370, 373, 376, 448, 535 resizing hash tables, 171 resource hierarchy solution, dining philosophers problem, 450–451 retracing, linked list, 75–76 retreat, attack See consensus problem reversal arrays, 117 Index ■ S–S linked list, 76–78 stacks, 128, 509 Reversi game, 304–305 right rotation, 279–281, 524–525, 526 Rivest, Ron, 412 rolls See dice rolls root at top, tree data structures, 228, 271 root merge, 287, 290 root split, 283, 284, 289 roots See also nodes of equations, 49–51, 496 root node, 228, 229, 231 rotations left, 279–281, 524–525 right, 278–281, 524–525, 526 round manhole covers, 465, 466 route ciphers, 403–404 row/column transposition cipher, 399–401, 402, 417 RowColumnTransposition example program, 540 row-major mapping, 84–85, 91 RSA algorithm, 412–415, 416, 417, 418 RtlMoveMemory, 18 Rule 1, 7, Rule 2, 7, 8–9 Rule 3, 7, Rule 4, 8, 9–10 Rule 5, 8, 10 runtime functions, 11–17 Fibonacci function compared to, 23, 489–490 graph, 18 list, in order of increasing speed of growth, 478 visualizing, 17 runtime performance See also Big O notation; sorting algorithms quicksort, 146–148 recursive algorithms, 187 sorting algorithms, 131 Tower of Hanoi puzzle, 193 traversal, 244 understanding, S sales program, nonzero bounds, 90 salesman problem See traveling salesman problem salvaging bad questions, 468 sample variance, one-dimensional array, 108, 501 satisfiability problem (SAT), 321–322, 424–425, 431 S-boxes, 409 Schneier, Bruce, 416 school bus, baseballs in, 467, 471 scratch array, 154 scytale method, 397, 399 searching abba dabba example, 389–391 brute-force, 299, 388, 389, 396, 410, 432, 448 nonindexed database, 448 random, 448 strings, 387–391 tools, 163 traversal and, 237 searching algorithms, 163–168 binary search described, 165–166, 169, 480 exercise, 168, 513 speed, 167 exercises, 168, 512–513 interpolation search described, 166–167, 169, 480 exercises, 168, 513 recursion, 168, 513 speed, 167, 168 linear search (exhaustive search) described, 164–165, 480 exercises, 168, 512 linked lists, 164 recursion, 168, 512 sorted arrays, 164 speed, 167 unsorted arrays, 86, 164–165 summary of concepts, 480 searching decision trees See decision trees bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 595 595 596 Index ■ S–S searching game trees See game trees secondary clustering, 177, 179 security through obscurity, 397, 398, 399 seed values, 26–28, 450 selections (combinations) defined, 209 with duplicates, 211–213 without duplicates, 213–214 with loops, 210–211 selectionsort algorithm arrays, 134–135 characteristics, 159 comparison, 159–160 defined, 479 insertionsort compared to, 82, 500 linked lists, 69–70 program, 160 queues, 129, 510 QueueSelectionsort example program, 510 stacks, 122–123, 129, 509–510 StackSelectionsort example program, 509 10 floating-point values, 161, 511 SelectKofN example program, 518 self-similar fractals, 193–194, 200, 223, 481 sender, 398 sentinels, 58–59, 246, 478 set method, 90, 94, 97 Shamir, Adi, 412 Shannon number, 298 Shor’s algorithm, 445 short circuiting, 308, 310, 323 shortest-path algorithms all-pairs shortest paths, 345–350, 353, 483, 534 label-correcting shortest paths, 344– 345, 352, 483, 533 label-setting shortest paths, 340–343, 344, 350, 352, 483, 533 stacks, 117 siblings, 228, 231 SierpDown, 199, 516 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 596 Sierpinski, Waclaw Franciszek, 201 Sierpinski carpet, 201, 224, 517–518 Sierpinski curves, 197–200, 224, 516, 517 Sierpinski gaskets, 200–201, 224, 517 SierpLeft, 199, 516 SierpRight, 199, 516 SierpUp, 199, 516–517 sieves general number field sieve, 445 sieve of Eratosthenes, 39–40, 53, 422, 478, 494 simple substitution cipher, 404, 407– 408 Simpson’s rule, 44 simulated annealing, 314, 320, 321, 483 single swap strategy, 313 singly linked lists, 56–63 six-sided dice See dice rolls 64-bit double-precision floating-point number, 188 slice_size array, 93–94 snapshot, 459–460, 486 snowflakes, Koch, 195, 223–224, 516 Social Security number, 292 socks, interview puzzles, 474, 551 solar system’s planets, linked list, 70–71, 82, 500 solutions See exercises sorted binary trees, 248, 250, 271, 278, 283 sorted chaining, 480, 513, 514 sorted hill climbing, 316, 320, 323, 483 sorted linked lists, 65–66 sorted trees See also balanced trees building, 245–247 defined, 12, 241, 482 nodes adding, 245–247 deleting, 248–250 finding, 247 sentinels, 246 SortedChaining example program, 513 sorting external, 155, 159, 480 Index ■ S–S linked lists copying lists, 67–68 insertionsort algorithm, 68–69 selectionsort algorithm, 69–70 stable, 155 tools, 132 topological, 355–359, 375, 484 train cars stack algorithms, 117–119 stack insertionsort algorithm, 129, 509 stack selectionsort algorithm, 129, 509–510 zero-time sort algorithm, 438, 446, 462, 547–548 sorting algorithms, 131–162 See also specific sorting algorithms characteristics/techniques, 159–160 exercises, 160–162, 510–512 O(N log N) algorithms, 138–155 O[N2] algorithms, 132–138 reasons for using, 131 runtime performance, 131 sub O(N log N) algorithms, 156–159 summary of concepts, 479–480 types, 131 space-filling curves, 200 spaghetti See dining philosophers problem spanning trees defined, 337 degree-constrained, 430 described, 337–338 maximum leaf spanning tree, 430 minimal, 329, 338–339, 350, 352, 483, 533 minimum degree spanning tree, 430 sparse arrays, 97–105 defined, 97, 479 deleting values, 104–105 find row or column, 100 getting values, 101 setting values, 101–104 sparse matrices add, 106–107, 109, 505–508 column-ordered, 107 multiply, 107, 109, 508–509 sparse triangular array, 109, 504 special alphabets, 397 specialized queues, 127–128 specialized tree algorithms, 256–270 animal game, 256–258, 274, 482, 521–522 expression evaluation, 258–260, 274–275 octtrees, 266, 482 quadtrees, 260–266, 482, 523 tries, 266–270, 275, 512, 523–524 special-purpose arrays See arrays speed, algorithms, 2, 477 splits node, 283–284, 291, 482 root, 283, 284, 289 spring-loaded stack, of plates, 111–112 sqrt N, 14, 17, 478 stable sorting, 155 stack algorithms, 117–123 StackInsertionsort example program, 509 stacks, 111–123 in arrays, 113–115 arrays compared to, 108 defined, 111, 187, 479 double, 115–117, 128, 509 exercises, 128–129, 509–510 insertionsort algorithm, 121–122, 128–129, 509 LIFO, 111, 117, 128, 333, 479 linked-list, 112–113 quicksort implementation, 149 reversal, 128, 509 selectionsort algorithm, 122–123, 129, 509–510 shortest-path algorithms, 117 summary of concepts, 479 uses, 111, 128, 479 StackSelectionsort example program, 509 standard deviation, one-dimensional array, 108, 501 bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 597 597 598 Index ■ T–T starvation, 444 state transition diagram, 382–383, 395, 396, 537, 538, 539 state transition table, 383 static, radio waves, 26 storing intermediate values, 218–219, 481 stride, 174, 176, 178, 179, 181, 514 string algorithms, 377–396 Boyer-Moore algorithm, 377, 388–391, 394, 484 DFAs, 381–386, 394, 395, 396, 421, 484, 537, 538, 539 edit distance algorithm, 3, 374, 391– 394, 396, 484, 540 exercises, 394–396, 537–540 matching parentheses, 378–380, 381, 394 parse trees, 380, 383, 387, 395, 396, 484, 539 regular expressions, 381–387, 394, 395, 484, 537, 538 string searching, 387–391 summary of concepts, 484 strongly connected network, 326, 328 stupid questions, interview puzzles, 472 sub O(N log N) algorithms See sorting algorithms subexpressions, 258–259, 384–386, 387 subset sum problem, 317–318, 427, 428, 431 substitution ciphers, 404–408, 416 substitution-permutation network cipher, 409–410, 416 subtrees, 229, 231 summary of algorithmic concepts See algorithmic concepts superposition, 445 SwapColumns example program, 540 swapping strategies, 313 SwapRowsAndColumns example program, 541 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 598 symmetric traversal See preorder traversal symmetrically threaded tree, 251 symmetric-key encryption, 412, 415 synchronization clocks, 460–461, 463–464, 486 generals’ attacks, 452 synchronized philosophers, 450, 451, 452, 462, 548 systolic arrays, 436–438, 446, 462, 485, 547 T tail recursion removal, 216–218 tails/heads See coin flips tall, thin trees, 147, 148, 233, 246, 247, 271, 277, 283, 287, 293 See also balanced trees tape drives, mergesort algorithm, 155 target_value, 12–13 task parallelism, 440 10 floating-point values, 161, 511 teraflop, 439 termination, consensus problem, 455 terminology cryptography, 398–399 networks, 325–328 trees, 227–231 test_node, 12–13 tetrahedral arrays, 109, 503–504 TextDisplay example program, 520 thin trees See tall, thin trees threaded trees, 250–256 building, 251–254 defined, 250 symmetrically, 251 traversal steps, 251, 255–256 using, 254–256 ThreadedTree example program, 521 threads defined, 250, 482 linked lists, 70–71, 82, 500 3CNF (three-term conjunctive normal form), 425 Index ■ T–T 3-node, 282 3SAT problem, 321–322, 425, 431 three-coloring maps, 362, 363, 364, 367, 375, 484, 535, 537 three-cycle problem, 432, 542–543 three-dimensional arrays 4×4×3, 92 graphical representation, 84 mapping, 85 row-major order, 85 three-dimensional shape, Monte Carlo integration, 54, 496 three-dimensional space, octtrees, 266, 482 three-partition problem, 431 three-person general and lieutenant problem, 454, 463 three-term conjunctive normal form (3CNF), 425 ticks systolic arrays, 437–438 zero-time sort algorithm, 438, 446, 462, 547–548 tic-tac-toe (naughts and crosses) exercise, 322, 531 initial moves and responses, 302–303 minimax strategy, 298–302 top variable, 56, 57 top-down B-tree, 291, 482 topological sorting, 355–359, 375, 484 tortoise-and-hare algorithm, 78–80, 82, 474, 500 totient function, Euler’s, 412, 413, 414 Tower of Hanoi puzzle, 119–121, 189– 193, 223, 515–516 TowerOfHanoi example program, 190, 515 train cars See sorting traitors See generals problem transposition ciphers, 399–404, 409, 417, 484 trapezoid rule, 43–45, 478 TrapezoidRule sample program, 43–44, 45, 46 traveling salesman problem (TSP), 16, 200, 320–321, 417, 423, 429, 431, 432 traversals, 237–244 defined, 237 depth-first, 243–244, 256, 272, 331– 334, 335, 350, 355, 483, 520 inorder, 240–241, 244, 245, 247, 250– 251, 254, 272, 273, 482, 520–521 postorder, 242–243, 244, 272, 482, 520 preorder, 238–240, 244, 271, 272, 331, 482, 520 runtimes, 244 searching and, 237 steps, threaded tree, 251, 255–256 tree theorems, inductive reasoning, 233, 481, 519–520 trees, 227–275 See also balanced trees; decision trees; nodes; subtrees binary defined, 230 facts, 232 fat, 233 properties, 231–233 sorted, 248, 250, 271, 278, 283 call tree, 146, 147, 154, 188, 189 complete binary trees in arrays, 139–140, 144, 145, 161, 236–237 B-trees, 287 defined, 230 sorted, 12–13 exercises, 271–275, 519–524 family tree analogy, 228 fat, 233 full, 229, 230, 231 height, 171, 229, 231 intuitive description, 227–228 knowledge, 256–258, 274, 482, 521 level/depth, 229, 231 networks compared to, 227, 322, 325, 332 ordered, 229, 231 parse, 380, 383, 387, 395, 396, 484, 539 properties, 481–482 bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 599 599 600 Index ■ U–U RandomTrees program, 27–28 recursive definition, 229 root at top, 228, 271 sorted adding nodes, 245–247 building, 245–247 defined, 12, 241, 482 deleting nodes, 248–250 finding nodes, 247 sentinels, 246 spanning trees defined, 337 degree-constrained, 430 described, 337–338 maximum leaf spanning tree, 430 minimal, 329, 338–339, 350, 352, 483, 533 minimum degree spanning tree, 430 specialized tree algorithms, 256–270 animal game, 256–258, 274, 482, 521–522 expression evaluation, 258–260, 274–275 octtrees, 266, 482 quadtrees, 260–266, 482, 523 tries, 266–270, 275, 512, 523–524 summary of concepts, 481–482 tall, thin, 147, 148, 233, 246, 247, 271, 277, 283, 287, 293 terminology, 227–231 threaded, 250–256 building, 251–254 defined, 250 symmetrically, 251 traversal steps, 251, 255–256 using, 254–256 unordered, 229, 273 XML and, 236 treesort algorithm, 247 triangles See Koch snowflakes; Sierpinski gaskets triangular arrays, 94–97 add, 109, 504 lower, 109, 502 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 600 multiply, 109, 505 sparse, 109, 504 sparse arrays compared to, 97, 98–99 tetrahedral arrays, 109, 503–504 upper, 94, 109, 502 tries (prefix trees), 266–270, 275, 512, 523–524 triple DES, 411, 416 trivia, 467, 468–469, 472, 486 TRNGs See true random-number generators true random numbers, 26 See also dice rolls true random-number generators (TRNGs), 25–26 See also PRNGs TSP See traveling salesman problem Turing, Alan, 421 Turing machines, 421–422, 424, 425 two generals problem, 452–453, 463, 485 2-3 trees, 282–287 adding values, 283–284 defined, 282, 482 deleting values, 284–287 exercise, 294, 527 variations, 291–293 2N, 16, 17, 478 2-node, 282 two-coloring maps, 360–361, 375, 484, 537 TwoDArray sample program, 90–91 two-dimensional arrays graphical representation, 84 int[,] numbers = new int[10, 20];, 84–85 nonzero local bounds, 90–91 two-dimensional matrices, multiply, 106 two-dimensional space See quadtrees U unbounded knapsack, 431 unbreakable encryption scheme, 418, 542 Index ■ V–Z undirected networks, 326, 327, 328, 329, 335, 337, 360, 545 uniform value distribution bucketsort, 157, 160, 162, 512 million integers, 162, 512 unordered trees, 229, 273 upper triangular arrays, 94, 109, 502 V validity, consensus problem, 456 values array, 91–94, 115, 154, 157 vehicle routing, 431 vertex coloring, 430 vertex cover, 431 vertices, 325, 430, 431 Via array, 346–349, 353, 534 Vigenère cipher, 405–407, 408, 411, 416, 417, 541 virtual backlinks, 369–370, 373 virtual computer, 381, 439, 484 visualizing runtime functions, 17 W waiter, dining philosophers problem, 451, 550 WAN, 267, 269, 270 WANE, 266–267, 269 WANT, 266, 267, 269 WANTED, 267, 269, 270 Warnsdorff’s heuristic, 209, 216, 225, 518 weakly connected network, 326, 328 While loop described, Euclid’s algorithm, 34–35 finding cells, 57–58 general recursion removal, 220 heaps, 143 iteration over linked list, 57 ordered hash table, 181 quicksort algorithm, 151–152 sentinels, 59 tail recursion, 218 threaded trees, 255, 256 “Why Are Manhole Covers Round?” article, 466 WISP, 266, 267 witnesses, Fermat, 40–41 work assignment, 370–371, 376, 426, 484, 536, 537 work related questions, interview puzzles, 472 working backwards, interview puzzles, 466, 467, 472, 486 worst-case performance, algorithms, X XML (eXtensible Markup Language) Beginning XML, 236 network data, 330 trees and, 236 XOR operator, 408, 409, 410 Z zeros, finding, 49–51 zero-time sort algorithm, 438, 446, 462, 547–548 bindex.indd CuuDuongThanCong.com 05:47:54:PM 07/10/2013 Page 601 601 bindex.indd 05:47:54:PM 07/10/2013 CuuDuongThanCong.com Page 602 ... performing a certain task A data structure is a way of arranging data to make solving a particular problem easier A data structure could be a way of arranging values in an array, a linked list that connects... such as triangular arrays and sparse arrays, that can save a program time and memory Chapter 5, “Stacks and Queues,” explains algorithms and data structures that let a program store and retrieve... 07/10 /2013 CuuDuongThanCong.com Page 602 Essential Algorithms A Practical Approach to Computer Algorithms Rod Stephens ffi rs.indd CuuDuongThanCong.com 05:53:16:PM 07/10 /2013 Page i Essential Algorithms: