Practical introduction to data structures WSLuZNyB92WPapRqR0hAF53MxaGcNBof pdf

A Practical Introduction to Data Structures and Algorithm Analysis Third Edition (C++ Version) Clifford A Shaffer Department of Computer Science Virginia Tech Blacksburg, VA 24061 January 19, 2010 Copyright c 2009-2010 by Clifford A Shaffer This document is made freely available for educational and other non-commercial use You may make copies of this file and redistribute it without charge You may extract portions of this document provided that the front page, including the title, author, and this notice are included Any commercial use of this document requires the written consent of the author The author can be reached at shaffer@cs.vt.edu Further information about this text is available at http://people.cs.vt.edu/˜shaffer/Book/ Contents Preface xiii I Preliminaries 1 Data Structures and Algorithms 1.1 A Philosophy of Data Structures 1.1.1 The Need for Data Structures 1.1.2 Costs and Benefits 1.2 Abstract Data Types and Data Structures 1.3 Design Patterns 1.3.1 Flyweight 1.3.2 Visitor 1.3.3 Composite 1.3.4 Strategy 1.4 Problems, Algorithms, and Programs 1.5 Further Reading 1.6 Exercises 4 12 13 14 15 16 17 19 21 Mathematical Preliminaries 2.1 Sets and Relations 2.2 Miscellaneous Notation 2.3 Logarithms 2.4 Summations and Recurrences 25 25 29 31 33 iii iv Contents 2.5 2.6 2.7 2.8 2.9 II Recursion Mathematical Proof Techniques 2.6.1 Direct Proof 2.6.2 Proof by Contradiction 2.6.3 Proof by Mathematical Induction Estimating Further Reading Exercises Algorithm Analysis 3.1 Introduction 3.2 Best, Worst, and Average Cases 3.3 A Faster Computer, or a Faster Algorithm? 3.4 Asymptotic Analysis 3.4.1 Upper Bounds 3.4.2 Lower Bounds 3.4.3 Θ Notation 3.4.4 Simplifying Rules 3.4.5 Classifying Functions 3.5 Calculating the Running Time for a Program 3.6 Analyzing Problems 3.7 Common Misunderstandings 3.8 Multiple Parameters 3.9 Space Bounds 3.10 Speeding Up Your Programs 3.11 Empirical Analysis 3.12 Further Reading 3.13 Exercises 3.14 Projects Fundamental Data Structures Lists, Stacks, and Queues 36 39 40 40 41 47 49 50 57 57 63 65 67 68 70 71 72 73 74 79 81 83 84 86 89 90 91 95 97 99 v Contents 4.1 4.2 4.3 4.4 4.5 4.6 4.7 Lists 4.1.1 Array-Based List Implementation 4.1.2 Linked Lists 4.1.3 Comparison of List Implementations 4.1.4 Element Implementations 4.1.5 Doubly Linked Lists Stacks 4.2.1 Array-Based Stacks 4.2.2 Linked Stacks 4.2.3 Comparison of Array-Based and Linked Stacks 4.2.4 Implementing Recursion Queues 4.3.1 Array-Based Queues 4.3.2 Linked Queues 4.3.3 Comparison of Array-Based and Linked Queues Dictionaries and Comparators Further Reading Exercises Projects Binary Trees 5.1 Definitions and Properties 5.1.1 The Full Binary Tree Theorem 5.1.2 A Binary Tree Node ADT 5.2 Binary Tree Traversals 5.3 Binary Tree Node Implementations 5.3.1 Pointer-Based Node Implementations 5.3.2 Space Requirements 5.3.3 Array Implementation for Complete Binary Trees 5.4 Binary Search Trees 5.5 Heaps and Priority Queues 5.6 Huffman Coding Trees 5.6.1 Building Huffman Coding Trees 100 104 107 118 120 121 127 127 128 131 131 135 136 139 142 142 152 153 156 159 159 162 163 165 169 169 175 177 178 188 195 196 vi Contents 5.7 5.8 5.9 III 5.6.2 Assigning and Using Huffman Codes Further Reading Exercises Projects Non-Binary Trees 6.1 General Tree Definitions and Terminology 6.1.1 An ADT for General Tree Nodes 6.1.2 General Tree Traversals 6.2 The Parent Pointer Implementation 6.3 General Tree Implementations 6.3.1 List of Children 6.3.2 The Left-Child/Right-Sibling Implementation 6.3.3 Dynamic Node Implementations 6.3.4 Dynamic “Left-Child/Right-Sibling” Implementation 6.4 K-ary Trees 6.5 Sequential Tree Implementations 6.6 Further Reading 6.7 Exercises 6.8 Projects Sorting and Searching Internal Sorting 7.1 Sorting Terminology and Notation 7.2 Three Θ(n2 ) Sorting Algorithms 7.2.1 Insertion Sort 7.2.2 Bubble Sort 7.2.3 Selection Sort 7.2.4 The Cost of Exchange Sorting 7.3 Shellsort 7.4 Mergesort 7.5 Quicksort 202 206 206 210 213 213 214 216 217 225 225 225 226 229 230 231 235 235 238 241 243 244 245 246 248 249 251 252 254 257 vii Contents 7.6 7.7 7.8 7.9 7.10 7.11 7.12 Heapsort Binsort and Radix Sort An Empirical Comparison of Sorting Algorithms Lower Bounds for Sorting Further Reading Exercises Projects 264 267 273 275 279 280 283 File Processing and External Sorting 8.1 Primary versus Secondary Storage 8.2 Disk Drives 8.2.1 Disk Drive Architecture 8.2.2 Disk Access Costs 8.3 Buffers and Buffer Pools 8.4 The Programmer’s View of Files 8.5 External Sorting 8.5.1 Simple Approaches to External Sorting 8.5.2 Replacement Selection 8.5.3 Multiway Merging 8.6 Further Reading 8.7 Exercises 8.8 Projects 287 288 290 291 294 297 305 306 309 312 314 319 319 323 Searching 9.1 Searching Unsorted and Sorted Arrays 9.2 Self-Organizing Lists 9.3 Bit Vectors for Representing Sets 9.4 Hashing 9.4.1 Hash Functions 9.4.2 Open Hashing 9.4.3 Closed Hashing 9.4.4 Analysis of Closed Hashing 9.4.5 Deletion 327 328 334 339 340 341 346 347 356 360 viii Contents 9.5 9.6 9.7 Further Reading Exercises Projects 361 362 365 10 Indexing 10.1 Linear Indexing 10.2 ISAM 10.3 Tree-based Indexing 10.4 2-3 Trees 10.5 B-Trees 10.5.1 B+ -Trees 10.5.2 B-Tree Analysis 10.6 Further Reading 10.7 Exercises 10.8 Projects 367 369 371 374 376 384 385 392 394 394 397 IV 399 Advanced Data Structures 11 Graphs 11.1 Terminology and Representations 11.2 Graph Implementations 11.3 Graph Traversals 11.3.1 Depth-First Search 11.3.2 Breadth-First Search 11.3.3 Topological Sort 11.4 Shortest-Paths Problems 11.4.1 Single-Source Shortest Paths 11.5 Minimum-Cost Spanning Trees 11.5.1 Prim’s Algorithm 11.5.2 Kruskal’s Algorithm 11.6 Further Reading 11.7 Exercises 11.8 Projects 401 402 406 410 414 415 417 420 421 424 426 429 431 431 434 Contents ix 12 Lists and Arrays Revisited 12.1 Multilists 12.2 Matrix Representations 12.3 Memory Management 12.3.1 Dynamic Storage Allocation 12.3.2 Failure Policies and Garbage Collection 12.4 Further Reading 12.5 Exercises 12.6 Projects 437 437 441 444 446 452 457 458 460 13 Advanced Tree Structures 13.1 Tries 13.2 Balanced Trees 13.2.1 The AVL Tree 13.2.2 The Splay Tree 13.3 Spatial Data Structures 13.3.1 The K-D Tree 13.3.2 The PR quadtree 13.3.3 Other Point Data Structures 13.3.4 Other Spatial Data Structures 13.4 Further Reading 13.5 Exercises 13.6 Projects 461 461 466 467 469 473 475 481 485 485 487 488 489 V 493 Theory of Algorithms 14 Analysis Techniques 14.1 Summation Techniques 14.2 Recurrence Relations 14.2.1 Estimating Upper and Lower Bounds 14.2.2 Expanding Recurrences 14.2.3 Divide and Conquer Recurrences 14.2.4 Average-Case Analysis of Quicksort 495 496 501 501 505 506 509 x Contents 14.3 14.4 14.5 14.6 Amortized Analysis Further Reading Exercises Projects 510 513 514 518 15 Lower Bounds 15.1 Introduction to Lower Bounds Proofs 15.2 Lower Bounds on Searching Lists 15.2.1 Searching in Unsorted Lists 15.2.2 Searching in Sorted Lists 15.3 Finding the Maximum Value 15.4 Adversarial Lower Bounds Proofs 15.5 State Space Lower Bounds Proofs 15.6 Finding the ith Best Element 15.7 Optimal Sorting 15.8 Further Reading 15.9 Exercises 15.10Projects 519 520 522 522 524 525 527 530 533 536 538 539 541 16 Patterns of Algorithms 16.1 Greedy Algorithms 16.2 Dynamic Programming 16.2.1 Knapsack Problem 16.2.2 All-Pairs Shortest Paths 16.3 Randomized Algorithms 16.3.1 Skip Lists 16.4 Numerical Algorithms 16.4.1 Exponentiation 16.4.2 Largest Common Factor 16.4.3 Matrix Multiplication 16.4.4 Random Numbers 16.4.5 Fast Fourier Transform 16.5 Further Reading 543 543 544 545 546 548 550 555 556 557 557 560 560 565 Contents 16.6 Exercises 16.7 Projects xi 565 566 17 Limits to Computation 17.1 Reductions 17.2 Hard Problems 17.2.1 The Theory of N P-Completeness 17.2.2 N P-Completeness Proofs 17.2.3 Coping with N P-Complete Problems 17.3 Impossible Problems 17.3.1 Uncountability 17.3.2 The Halting Problem Is Unsolvable 17.4 Further Reading 17.5 Exercises 17.6 Projects 567 568 573 574 579 583 587 588 591 595 595 598 VI 599 APPENDIX A Utility Functions 601 Bibliography 603 Index 609 ... Preliminaries 1 Data Structures and Algorithms 1.1 A Philosophy of Data Structures 1.1.1 The Need for Data Structures 1.1.2 Costs and Benefits 1.2 Abstract Data Types and Data Structures 1.3 Design... covers most standard data structures, but not all A few data structures that are not widely adopted are included to illustrate important principles Some relatively new data structures that should... primary goals The first is to present the commonly used data structures These form a programmer’s basic data structure “toolkit.” For many problems, some data structure in the toolkit provides a good