Building Skills in Python Release 2.6.2 Steven F Lott January 03, 2010 CONTENTS I Front Matter Preface 1.1 Why Read This Book? 1.2 Audience 1.3 Organization of This Book 1.4 Limitations 1.5 Programming Style 1.6 Conventions Used in This Book 1.7 Acknowledgements II Language Basics 5 9 10 11 Background and History 15 2.1 History 15 2.2 Features of Python 15 2.3 Comparisons 16 Python Installation 3.1 Windows Installation 3.2 Macintosh Installation 3.3 GNU/Linux and UNIX Overview 3.4 “Build from Scratch” Installation 21 21 24 25 28 Getting Started 4.1 Command-Line Interaction 4.2 The IDLE Development Environment 4.3 Script Mode 4.4 Getting Help 4.5 Syntax Formalities 4.6 Exercises 4.7 Other Tools 4.8 Style Notes: Wise Choice of File Names 31 31 34 36 40 41 42 44 45 Statement) 47 47 50 53 54 Simple Numeric Expressions and Output 5.1 Seeing Output with the print() Function (or print 5.2 Numeric Types and Operators 5.3 Numeric Conversion (or “Factory”) Functions 5.4 Built-In Math Functions i 5.5 5.6 Expression Exercises Expression Style Notes Advanced Expressions 6.1 Using Modules 6.2 The math Module 6.3 The random Module 6.4 Advanced Expression Exercises 6.5 Bit Manipulation Operators 6.6 Division Operators 56 60 61 61 61 63 64 66 68 Variables, Assignment and Input 7.1 Variables 7.2 The Assignment Statement 7.3 Input Functions 7.4 Multiple Assignment Statement 7.5 The del Statement 7.6 Interactive Mode Revisited 7.7 Variables, Assignment and Input Function Exercises 7.8 Variables and Assignment Style Notes 71 71 73 75 78 78 79 80 81 Truth, Comparison and Conditional Processing 8.1 Truth and Logic 8.2 Comparisons 8.3 Conditional Processing: the if Statement 8.4 The pass Statement 8.5 The assert Statement 8.6 The if-else Operator 8.7 Condition Exercises 8.8 Condition Style Notes 83 83 85 88 90 91 92 93 94 Loops and Iterative Processing 9.1 Iterative Processing: For All and There Exists 9.2 Iterative Processing: The for Statement 9.3 Iterative Processing: The while Statement 9.4 More Iteration Control: break and continue 9.5 Iteration Exercises 9.6 Condition and Loops Style Notes 9.7 A Digression 95 95 96 97 98 100 103 104 Statements 10 Functions 10.1 Semantics 10.2 Function Definition: The def and return 10.3 Function Use 10.4 Function Varieties 10.5 Some Examples 10.6 Hacking Mode 10.7 More Function Definition Features 10.8 Function Exercises 10.9 Object Method Functions 10.10 Functions Style Notes 107 107 109 110 111 112 113 115 118 121 122 11 Additional Notes On Functions 125 11.1 Functions and Namespaces 125 11.2 The global Statement 127 ii 11.3 Call By Value and Call By Reference 127 11.4 Function Objects 129 III Data Structures 131 12 Sequences: Strings, Tuples and Lists 12.1 Sequence Semantics 12.2 Overview of Sequences 12.3 Exercises 12.4 Style Notes 135 135 136 139 139 13 Strings 13.1 String Semantics 13.2 String Literal Values 13.3 String Operations 13.4 String Comparison Operations 13.5 String Statements 13.6 String Built-in Functions 13.7 String Methods 13.8 String Modules 13.9 String Exercises 13.10 Digression on Immutability of Strings 141 141 141 143 146 146 147 148 151 152 153 14 Tuples 14.1 Tuple Semantics 14.2 Tuple Literal Values 14.3 Tuple Operations 14.4 Tuple Comparison Operations 14.5 Tuple Statements 14.6 Tuple Built-in Functions 14.7 Tuple Exercises 14.8 Digression on The Sigma Operator 155 155 155 156 157 157 158 160 161 15 Lists 15.1 List Semantics 15.2 List Literal Values 15.3 List Operations 15.4 List Comparison Operations 15.5 List Statements 15.6 List Built-in Functions 15.7 List Methods 15.8 Using Lists as Function Parameter 15.9 List Exercises Defaults 163 163 163 164 164 165 166 167 169 170 Defaults 175 175 176 176 178 178 179 180 181 182 16 Mappings and Dictionaries 16.1 Dictionary Semantics 16.2 Dictionary Literal Values 16.3 Dictionary Operations 16.4 Dictionary Comparison Operations 16.5 Dictionary Statements 16.6 Dictionary Built-in Functions 16.7 Dictionary Methods 16.8 Using Dictionaries as Function Parameter 16.9 Dictionary Exercises iii 16.10 Advanced Parameter Handling For Functions 184 17 Sets 17.1 17.2 17.3 17.4 17.5 17.6 17.7 17.8 17.9 Set Semantics Set Literal Values Set Operations Set Comparison Operators Set Statements Set Built-in Functions Set Methods Using Sets as Function Parameter Defaults Set Exercises 187 187 187 188 190 191 191 192 194 195 18 Exceptions 18.1 Exception Semantics 18.2 Basic Exception Handling 18.3 Raising Exceptions 18.4 An Exceptional Example 18.5 Complete Exception Handling and 18.6 Exception Functions 18.7 Exception Attributes 18.8 Built-in Exceptions 18.9 Exception Exercises 18.10 Style Notes 18.11 A Digression The finally Clause 199 199 200 203 204 206 206 207 208 210 211 212 19 Iterators and Generators 19.1 Iterator Semantics 19.2 Generator Function Semantics 19.3 Defining a Generator Function 19.4 Generator Functions 19.5 Generator Statements 19.6 Iterators Everywhere 19.7 Generator Function Example 19.8 Generator Exercises 213 213 214 215 216 217 217 218 219 20 Files 20.1 File Semantics 20.2 File Organization and Structure 20.3 Additional Background 20.4 Built-in Functions 20.5 File Statements 20.6 File Methods 20.7 Several Examples 20.8 File Exercises 221 221 222 223 224 226 226 228 232 21 Functional Programming with Collections 21.1 Lists of Tuples 21.2 List Comprehensions 21.3 Sequence Processing Functions: map(), filter() and reduce() 21.4 Advanced List Sorting 21.5 The Lambda 21.6 Multi-Dimensional Arrays or Matrices 21.7 Exercises 235 235 236 239 242 244 246 248 iv 22 Advanced Mapping Techniques 251 22.1 Default Dictionaries 251 22.2 Inverting a Dictionary 252 22.3 Exercises 253 IV Data + Processing = Objects 255 23 Classes 23.1 Semantics 23.2 Class Definition: the class Statement 23.3 Creating and Using Objects 23.4 Special Method Names 23.5 Some Examples 23.6 Object Collaboration 23.7 Class Definition Exercises 259 259 262 263 264 266 269 271 24 Advanced Class Definition 24.1 Inheritance 24.2 Polymorphism 24.3 Built-in Functions 24.4 Collaborating with max(), min() and sort() 24.5 Initializer Techniques 24.6 Class Variables 24.7 Static Methods and Class Method 24.8 Design Approaches 24.9 Advanced Class Definition Exercises 24.10 Style Notes 287 287 292 294 296 296 297 299 299 301 303 25 Some Design Patterns 25.1 Factory 25.2 State 25.3 Strategy 25.4 Design Pattern Exercises 307 307 310 313 315 26 Creating or Extending Data Types 26.1 Semantics of Special Methods 26.2 Basic Special Methods 26.3 Special Attribute Names 26.4 Numeric Type Special Methods 26.5 Collection Special Method Names 26.6 Collection Special Method Names for Iterators and Iterable 26.7 Collection Special Method Names for Sequences 26.8 Collection Special Method Names for Sets 26.9 Collection Special Method Names for Mappings 26.10 Mapping Example 26.11 Iterator Examples 26.12 Extending Built-In Classes 26.13 Special Method Name Exercises 319 320 321 322 322 327 329 330 331 332 333 334 336 336 27 Attributes, Properties and Descriptors 27.1 Semantics of Attributes 27.2 Properties 27.3 Descriptors 27.4 Attribute Handling Special Method Names 343 343 344 346 348 v 27.5 Attribute Access Exercises 349 28 Decorators 28.1 Semantics of Decorators 28.2 Built-in Decorators 28.3 Defining Decorators 28.4 Defining Complex Decorators 28.5 Decorator Exercises 351 351 352 354 355 356 29 Managing Contexts: the with Statement 29.1 Semantics of a Context 29.2 Using a Context 29.3 Defining a Context Manager Function 29.4 Defining a Context Manager Class 29.5 Context Manager Exercises 357 357 358 358 360 361 V Components, Modules and Packages 363 30 Modules 30.1 Module Semantics 30.2 Module Definition 30.3 Module Use: The import Statement 30.4 Finding Modules: The Path 30.5 Variations on An import Theme 30.6 The exec Statement 30.7 Module Exercises 30.8 Style Notes 367 367 368 370 372 373 375 375 377 31 Packages 31.1 Package Semantics 31.2 Package Definition 31.3 Package Use 31.4 Package Exercises 31.5 Style Notes 379 379 380 381 381 381 32 The 32.1 32.2 32.3 Python Library 383 Overview of the Python Library 383 Most Useful Library Sections 385 Library Exercises 393 33 Complex Strings: the re Module 33.1 Semantics 33.2 Creating a Regular Expression 33.3 Using a Regular Expression 33.4 Regular Expression Exercises 395 395 396 397 399 34 Dates and Times: the time and datetime Modules 34.1 Semantics: What is Time? 34.2 Some Class Definitions 34.3 Creating a Date-Time 34.4 Date-Time Calculations and Manipulations 34.5 Presenting a Date-Time 34.6 Formatting Symbols 34.7 Time Exercises 401 401 403 404 405 407 408 409 vi 34.8 Additional time Module Features 410 35 File Handling Modules 35.1 The os.path Module 35.2 The os Module 35.3 The fileinput Module 35.4 The glob and fnmatch Modules 35.5 The tempfile Module 35.6 The shutil Module 35.7 The File Archive Modules: tarfile and zipfile 35.8 The sys Module 35.9 Additional File-Processing Modules 35.10 File Module Exercises 36 File 36.1 36.2 36.3 36.4 36.5 36.6 36.7 36.8 36.9 Formats: CSV, Tab, XML, Logs and Others Overview Comma-Separated Values: The csv Module Tab Files: Nothing Special Property Files and Configuration (or INI ) Files: The ConfigParser Fixed Format Files, A COBOL Legacy: The codecs Module XML Files: The xml.etree and xml.sax Modules Log Files: The logging Module File Format Exercises The DOM Class Hierarchy 37 Programs: Standing Alone 37.1 Kinds of Programs 37.2 Command-Line Programs: Servers 37.3 The optparse Module 37.4 Command-Line Examples 37.5 Other Command-Line Features 37.6 Command-Line Exercises 37.7 The getopt Module and Batch Processing 38 Architecture: Clients, Servers, the Internet and 38.1 About TCP/IP 38.2 The World Wide Web and the HTTP protocol 38.3 Writing Web Clients: The urllib2 Module 38.4 Writing Web Applications 38.5 Sessions and State 38.6 Handling Form Inputs 38.7 Web Services 38.8 Client-Server Exercises 38.9 Socket Programming VI Projects the World Wide 411 413 414 416 417 418 419 419 423 424 425 Module 427 427 428 431 432 434 436 441 446 446 451 451 453 455 458 459 461 461 Web 465 465 466 467 469 477 478 480 485 491 499 39 Areas of the Flag 503 39.1 Basic Red, White and Blue 503 39.2 The Stars 504 40 The Date of Easter 507 40.1 Algorithm E 507 40.2 Algorithm J 508 vii 40.3 40.4 40.5 40.6 40.7 40.8 40.9 40.10 40.11 40.12 40.13 40.14 Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm Algorithm A B O P F G R L RD Y M D 508 509 509 510 511 511 512 512 512 513 513 514 41 Musical Pitches 41.1 Equal Temperament 41.2 Overtones 41.3 Circle of Fifths 41.4 Pythagorean Tuning 41.5 Five-Tone Tuning 515 516 517 517 518 519 42 Bowling Scores 43 Mah 43.1 43.2 43.3 43.4 43.5 43.6 43.7 43.8 521 523 523 525 526 528 529 531 533 536 44 Chess Game Notation 44.1 Algebraic Notation 44.2 Algorithms for Resolving Moves 44.3 Descriptive Notation 44.4 Game State 44.5 PGN Processing Specifications 539 539 543 546 546 547 VII Jongg Hands Tile Class Hierarchy Wall Class TileSet Class Hierarchy Hand Class Some Test Cases Hand Scoring - Points Hand Scoring - Doubles Limit Hands Back Matter 45 Bibliography 45.1 Use Cases 45.2 Computer Science 45.3 Design Patterns 45.4 Languages 45.5 Problem Domains 549 551 551 551 551 551 551 46 Indices and Tables 553 47 Production Notes 555 Bibliography 557 viii