Thông tin tài liệu
CuuDuongThanCong.com SECOND EDITION Algorithms in a Nutshell 2E George T Heineman, Gary Pollice, and Stanley Selkow Boston CuuDuongThanCong.com Algorithms in a Nutshell 2E, Second Edition by George T Heineman, Gary Pollice, and Stanley Selkow Copyright © 2010 George Heineman, Gary Pollice and Stanley Selkow All rights re‐ served Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://safaribooksonline.com) For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com Editor: Mary Treseler Production Editor: FIX ME! Copyeditor: FIX ME! Proofreader: FIX ME! January -4712: Indexer: FIX ME! Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Rebecca Demarest Second Edition Revision History for the Second Edition: 2015-07-27: Early release revision See http://oreilly.com/catalog/errata.csp?isbn=0636920032885 for release details Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc !!FILL THIS IN!! and related trade dress are trade‐ marks of O’Reilly Media, Inc Many of the designations used by manufacturers and sellers to distinguish their prod‐ ucts are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc was aware of a trademark claim, the designations have been printed in caps or initial caps While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein ISBN: 063-6-920-03288-5 [?] CuuDuongThanCong.com Table of Contents Thinking Algorithmically Understand the Problem Naive Solution Intelligent Approaches Greedy Divide and Conquer Parallel Approximation Generalization Summary 4 5 The Mathematics of Algorithms Size of a Problem Instance Rate of Growth of Functions Analysis in the Best, Average, and Worst Cases Worst Case Average Case Best Case Performance Families Constant Behavior Log n Behavior Sublinear O(nd) Behavior for d < Linear Performance n log n Performance Quadratic Performance Less Obvious Performance Computations Exponential Performance Benchmark Operations 10 15 18 18 19 20 20 21 23 23 27 28 30 33 33 iii CuuDuongThanCong.com Lower and Upper Bounds References 36 36 Algorithm Building Blocks 37 Algorithm Template Format Name Input/Output Context Solution Analysis Variations Pseudocode Template Format Empirical Evaluation Format Floating-Point Computation Performance Rounding Error Comparing Floating Point Values Special Quantities Example Algorithm Name and Synopsis Input/Output Context Solution Analysis Common Approaches Greedy Divide and Conquer Dynamic Programming References 37 38 38 38 38 38 39 39 40 40 41 41 43 44 45 45 46 46 46 49 49 49 50 51 56 Sorting Algorithms 57 Overview Terminology Representation Comparable Elements Stable Sorting Criteria for Choosing a Sorting Algorithm Transposition Sorting Insertion Sort Context Solution iv | Table of Contents CuuDuongThanCong.com 57 57 58 59 60 61 61 61 63 63 Analysis Selection Sort Heap Sort Context Solution Analysis Variations Partition-based Sorting Context Solution Analysis Variations Sorting Without Comparisons Bucket Sort Solution Analysis Variations Sorting with Extra Storage Merge Sort Input/Output Solution Analysis Variations String Benchmark Results Analysis Techniques References 65 66 67 72 73 74 74 74 80 80 81 81 83 83 86 88 89 90 90 92 92 93 94 95 98 99 Searching 101 Sequential Search Input/Output Context Solution Analysis Binary Search Input/Output Context Solution Analysis Variations Hash-based Search Input/Output 102 103 103 104 105 106 106 107 107 108 110 111 113 Table of Contents CuuDuongThanCong.com | v Context Solution Analysis Variations Bloom Filter Input/Output Context Solution Analysis Binary Search Tree Input/Output Context Solution Analysis Variations References 114 117 119 122 127 129 129 129 131 132 133 133 135 146 146 146 Graph Algorithms 149 Graphs Data Structure Design Depth-First Search Input/Output Context Solution Analysis Variations Breadth-First Search Input/Output Context Solution Analysis Single-Source Shortest Path Input/Output Solution Analysis Dijkstra’s Algorithm For Dense Graphs Variations Comparing Single Source Shortest Path Options Benchmark data Dense graphs Sparse graphs vi | Table of Contents CuuDuongThanCong.com 151 154 155 160 161 161 163 164 164 167 168 168 169 169 172 172 174 174 177 180 181 181 182 All Pairs Shortest Path Input/Output Solution Analysis Minimum Spanning Tree Algorithms Solution Analysis Variations Final Thoughts on Graphs Storage Issues Graph Analysis References 183 186 186 188 188 191 192 192 192 192 193 194 Path Finding in AI 195 Game Trees Minimax Input/Output Context Solution Analysis NegMax Solution Analysis AlphaBeta Solution Analysis Search Trees Representing State Calculate available moves Using Heuristic Information Maximum Expansion Depth Depth-First Search Input/Output Context Solution Analysis Breadth-First Search Input/Output Context Solution Analysis 196 199 202 202 203 205 206 208 210 210 214 215 217 220 221 221 223 223 225 225 225 227 230 232 232 233 234 Table of Contents CuuDuongThanCong.com | vii A*Search Input/Output Context Solution Analysis Variations Comparing Search Tree Algorithms References 234 236 236 239 243 246 247 251 Network Flow Algorithms 255 Network Flow Maximum Flow Input/Output Solution Analysis Optimization Related Algorithms Bipartite Matching Input/Output Solution Analysis Reflections on Augmenting Paths Minimum Cost Flow Transshipment Solution Transportation Solution Assignment Solution Linear Programming References 257 259 261 262 267 268 270 270 271 271 274 274 279 280 280 283 283 283 283 283 285 Computational Geometry 287 Classifying Problems Input data Computation Nature of the task Assumptions Convex Hull Convex Hull Scan Input/Output viii | CuuDuongThanCong.com Table of Contents 288 288 290 291 291 291 293 295 Context Solution Analysis Variations Computing Line Segment Intersections LineSweep Input/Output Context Solution Analysis Variations Voronoi Diagram Input/Output Solution Analysis References 295 295 297 299 302 303 306 306 307 310 313 313 321 322 327 328 10 Spatial Tree Structures 329 Nearest Neighbor queries Range Queries Intersection Queries Spatial Tree Structures KD-Tree Quad Tree R-Tree Nearest Neighbor Input/Output Context Solution Analysis Variations Range Query Input/Output Context Solution Analysis QuadTrees Input/Output Solution Analysis Variations 330 331 331 332 332 333 334 335 337 338 338 340 347 347 349 350 350 351 355 358 359 362 363 Table of Contents CuuDuongThanCong.com | ix threshold (which we set to 10−9), unexpected behavior would often occur Eliminating the threshold entirely won’t solve the problem, ei‐ ther We ultimately resorted to statistically checking the results of these algorithms, rather than seeking absolute and definitive answers for all cases Table 12-7 summarizes the algorithms presented in Chapter Each algorithm shares the challenges in working with two-dimensional geometric structures and accurately performing geometric computa‐ tions Table 12-7 Chapter 9: Computational geometry Algorithm Best Convex Hull Scan n Line Sweep Average Worst Concepts Page n log n n log n Array, Greedy “Convex Hull Scan” on page 293 (n+k) log n (n+k) log n n2 Voronoi Diagram n log n n log n Priority Queue, Binary Tree “LineSweep” on page 303 n log n Line Sweep, Priority Queue, Binary Tree “Voronoi Diagram” on page 313 Principle: Accept Approximate Solution When Possible In many circumstances, an approximate result is acceptable if it can be computed much faster than an accurate result and it has a known error from the correct result The Knapsack unbounded problem provides such a scenario, since the approximation is no worse than 50% of the actual result These approximations can use randomness to compute an estimate of an actual answer, as we saw with the example for counting the number of solutions to the “N Queens Problem.” Use this approach when you know that repeated trials increase the preci‐ sion of the estimate A Bloom Filter is carefully designed so it can return false positives, but never false negatives, when searching for an element in a collec‐ tion At first glance, it may seem useless to have an algorithm that returns an incorrect answer But a Bloom Filter can dramatically re‐ duce the execution time of searching algorithms involving secondary storage or database systems When it returns negative, it truly means that the element does not exist in the collection, so there is no need to expend the costly resources Of course, it might mean that sometimes 410 | CuuDuongThanCong.com Chapter 12: Epilogue the Bloom Filter allows a search to continue which will fail, but this won’t affect the correctness of the overall application Principle: Add Parallelism to Increase Performance The algorithms presented in this book compute their results assuming a single, sequential computer If you can identify sub-problems that can be independently computed, you might be able to design a multithreaded solution using the available resources provided by modern computers For instance, “Parallel Algorithms” on page 386 showed how to parallelize Quicksort to achieve a nice speed up What other algorithms in this book can benefit from parallelism? Recall that Con‐ vex Hull Scan, has a sorting sub-step followed by two independent problems: constructing the lower partial hull and the upper partial hull Each of these tasks can be parallelized to achieve improved per‐ formance Table 12-8 shows the impressive speed up (review the algs.model.problems.convexhull.parallel code in the reposito‐ ry) Despite the impressive performance, the algorithm still performs in O(n log n) time, though with better constants Table 12-8 Performance improvements of multithreaded Convex Hull Scan n single-threaded helper thread helpers helpers helpers 2,048 0.8571 0.5000 0.6633 0.5204 0.6020 4,096 1.5204 0.7041 0.7041 0.7755 0.7857 8,192 3.3163 0.9592 1.0306 1.0306 1.0816 16,384 7.3776 1.6327 1.6327 1.5612 1.6939 32,768 16.3673 3.0612 2.8980 2.9694 3.1122 65,536 37.1633 5.8980 6.0102 6.0306 6.0408 131,072 94.2653 13.8061 14.3776 14.1020 14.5612 262,144 293.2245 37.0102 37.5204 37.5408 38.2143 524,288 801.7347 90.7449 92.1939 91.1633 91.9592 197.4592 198.6939 198.0306 200.5612 1,048,576 1890.5612 Most serial algorithms cannot achieve theoretic maximal speedup be‐ cause only part of the algorithm can be parallelized among multiple threads; this is known as Amdahl’s law Don’t try to use as many threads as possible in a solution Adding multiple helper threads re‐ quires more complicated programs than adding a single helper thread Principle: Add Parallelism to Increase Performance CuuDuongThanCong.com | 411 So with only a small increase in complexity, using a single helper thread can provide noticeable improvement However, not every algorithm can be improved with parallelism In the kd-tree Nearest Neighbor, for example, there may be double re‐ cursions as the algorithm seeks to find the closest point in the collec‐ tion to a target point Parallelizing these separate method invocations will slow down the overall performance because of the need to syn‐ chronize these helper threads so they both complete together 412 | CuuDuongThanCong.com Chapter 12: Epilogue APPENDIX A Benchmarking Each algorithm in this book is accompanied by data about its perfor‐ mance Because it’s important to use the right benchmarks to get ac‐ curate performance, and you have questions or doubts about the val‐ idity of our approach, we present our infrastructure to evaluate algo‐ rithm performance in this appendix We try to explain the precise means by which empirical data is computed, in order to enable you both to verify that the results are accurate and to understand where the assumptions are appropriate or inappropriate given the context in which the algorithm is intended to be used There are numerous ways by which algorithms can be analyzed Chapter presented a theoretical, formal treatment, introducing the concepts of worst-case and average-case analysis These theoretic re‐ sults can be empirically evaluated in some cases, though not all For example, consider evaluating the performance of an algorithm to sort 20 numbers There are 2.43*1018 permutations of these 20 numbers, and one cannot simply exhaustively evaluate each of these permuta‐ tions to compute the average case Additionally, one cannot compute the average by measuring the time to sort all of these permutations We must rely on statistical measures to assure ourselves that we have properly computed the expected performance time of the algorithm Statistical Foundation This chapter focuses on essential points to evaluating the performance of the algorithms Interested readers should consult any of the large number of available textbooks on statistics for more information on 413 CuuDuongThanCong.com the relevant statistical information used to produce the empirical measurements in this book To compute the performance of an algorithm, we construct a suite of T independent trials for which the algorithm is executed Each trial is intended to execute an algorithm on an input problem of size n Some effort is made to ensure that these trials are all reasonably equivalent for the algorithm When the trials are actually identical, the intent of the trial is to quantify the variance of the underlying implementation of the algorithm This may be suitable, for example, if it is too costly to compute a large number of independent equivalent trials The suite is executed and millisecond-level timings are taken before and after the observable behavior When the code is written in Java, the system garbage collector is invoked immediately prior to launching the trial; although this effort can’t guarantee that the garbage collector does not execute during the trial, it may reduce this risk of spending extra time unrelated to the algorithm From the full set of T recorded times, the best and worst performing times are discarded as being “outliers.” The remaining T−2 time records are averaged, and a stan‐ dard deviation is computed using the following formula: where xi is the time for an individual trial and x is the average of the T−2 trials Note here that n is equal to T_−2, so the denominator within the square root is _T−3 Calculating averages and standard deviations will help predict future performance, based on Table A-1, which shows the probability (between and 1) that the actual value will be within the range [x-k*σ, x+k*σ], where σ represents the standard deviation computed in the equation just shown The probability values become confidence intervals that declare the confidence we have in a predic‐ tion Table A-1 Standard deviation table k Probability 0.6827 0.9545 0.9973 0.9999 414 | CuuDuongThanCong.com Appendix A: Benchmarking For example, in a randomized trial, it is expected that 68.27% of the time the result will fall within the range [x-σ, x+σ] When reporting results, we never present numbers with greater than four decimal digits of accuracy, so we don’t give the mistaken impres‐ sion that we believe the accuracy of our numbers extends any farther When the computed fifth and greater digits falls in the range [0, 49,999], these digits are simply truncated; otherwise, the fourth digit is incremented to reflect the proper rounding This process will con‐ vert a computation such as 16.897986 into the reported number 16.8980 Example Assume we wanted to benchmark the addition of the numbers from to n An experiment is designed to measure the times for n=8,000,000 to n=16,000,000 in increments of two million Because the problem is identical for n and doesn’t vary, we execute for 30 trials to eliminate as much variability as possible The hypothesis is that the time to complete the sum will vary directly in relation to n We show three programs that solve this problem—in Java, C, and Python—and present the benchmark infrastructure by showing how it is used Java benchmarking solutions On Java test cases, the current system time (in milliseconds) is deter‐ mined immediately prior to, and after, the execution of interest The code in Example A-1 measures the time it takes to complete the task In a perfect computer, the 30 trials should all require exactly the same amount of time Of course this is unlikely to happen, because modern operating systems have numerous background processing tasks that share the same CPU on which the performance code executes Example A-1 Java example to time execution of task public class Main { public static void main (String[] args) { TrialSuite ts = new TrialSuite(); for (long len = 8000000; len $RESULTS TRIALS=$((TRIALS-1)) done # compute average/stdev RES=`cat $RESULTS | $CODE/eval` rm -f $RESULTS done compare.sh makes use of a small C program, eval, that computes the average and standard deviation using the method described at the start of this chapter This compare.sh script is repeatedly executed by a manager script, suiteRun.sh, that iterates over the desired input prob‐ lem sizes specified within the config.rc file, as shown in Example A-5 Example A-5 suiteRun.sh benchmarking script #!/bin/bash CODE=`dirname $0` # if no args then use default config file, otherwise expect it if [ $# -eq ] then CONFIG="config.rc" else CONFIG=$1 echo "Using configuration file $CONFIG " Example CuuDuongThanCong.com | 419 fi # export so it will be picked up by compare.sh export CONFIG # pull out information if [ -f $CONFIG ] then BINS=`grep "BINS=" $CONFIG | cut -f2- -d'='` TRIALS=`grep "TRIALS=" $CONFIG | cut -f2- -d'='` LOW=`grep "LOW=" $CONFIG | cut -f2- -d'='` HIGH=`grep "HIGH=" $CONFIG | cut -f2- -d'='` INCREMENT=`grep "INCREMENT=" $CONFIG | cut -f2- -d'='` else echo "Configuration file ($CONFIG) unable to be found." exit −1 fi # headers HB=`echo $BINS | tr ' ' ','` echo "n,$HB"# compare trials on sizes from LOW through HIGH SIZE=$LOW REPORT=/tmp/Report.$$ while [ $SIZE -le $HIGH ] # one per $BINS entry $CODE/compare.sh $SIZE $TRIALS | awk 'BEGIN{p=0} \ {if(p) { print $0; }} \ /Host:/{p=1}' | cut -d' ' -f2 > $REPORT # concatenate with, all entries ONLY the average The stdev is # going to be ignored # -VALS=`awk 'BEGIN{s=""}\ {s = s "," $0 }\ END{print s;}' $REPORT` rm -f $REPORT echo $SIZE $VALS # $INCREMENT can be "+ NUM" or "* NUM", it works in both cases SIZE=$(($SIZE$INCREMENT)) done Python benchmarking solutions The Python code in this section measures the performance of com‐ puting the addition problem It uses the timeit module, a standard for measuring execution time of Python code fragments and entire pro‐ grams 420 | CuuDuongThanCong.com Appendix A: Benchmarking Example A-6 Python example to time execution of task import timeit def performance(): """Demonstrate execution performance.""" n = 8000000 numTrials = 10 print ("n", "Add time") while n
Ngày đăng: 30/08/2020, 07:27
Xem thêm:
