FIFTH EDITION Download from Wow! eBook Learning Python Mark Lutz Beijing • Cambridge • Farnham • Kưln • Sebastopol • Tokyo Learning Python, Fifth Edition by Mark Lutz Copyright © 2013 Mark Lutz All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://my.safaribooksonline.com) For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com Editor: Rachel Roumeliotis Production Editor: Christopher Hearse Copyeditor: Rachel Monaghan Proofreader: Julie Van Keuren June 2013: Indexer: Lucie Haskins Cover Designer: Randy Comer Interior Designer: David Futato Illustrator: Rebecca Demarest Fifth Edition Revision History for the Fifth Edition: 2013-06-07 First release See http://oreilly.com/catalog/errata.csp?isbn=9781449355739 for release details Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc Learning Python, 5th Edition, the image of a wood rat, and related trade dress are trademarks of O’Reilly Media, Inc Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein ISBN: 978-1-449-35573-9 [QG] 1370970520 To Vera You are my life Table of Contents Preface xxxiii Part I Getting Started A Python Q&A Session Why Do People Use Python? Software Quality Developer Productivity Is Python a “Scripting Language”? OK, but What’s the Downside? Who Uses Python Today? What Can I Do with Python? Systems Programming GUIs Internet Scripting Component Integration Database Programming Rapid Prototyping Numeric and Scientific Programming And More: Gaming, Images, Data Mining, Robots, Excel How Is Python Developed and Supported? Open Source Tradeoffs What Are Python’s Technical Strengths? It’s Object-Oriented and Functional It’s Free It’s Portable It’s Powerful It’s Mixable It’s Relatively Easy to Use It’s Relatively Easy to Learn It’s Named After Monty Python 5 10 11 11 11 12 12 13 13 14 15 15 16 16 17 17 18 19 19 20 20 v How Does Python Stack Up to Language X? Chapter Summary Test Your Knowledge: Quiz Test Your Knowledge: Answers 21 22 23 23 How Python Runs Programs 27 Introducing the Python Interpreter Program Execution The Programmer’s View Python’s View Execution Model Variations Python Implementation Alternatives Execution Optimization Tools Frozen Binaries Future Possibilities? Chapter Summary Test Your Knowledge: Quiz Test Your Knowledge: Answers 27 28 28 30 33 33 37 39 40 40 41 41 How You Run Programs 43 The Interactive Prompt Starting an Interactive Session The System Path New Windows Options in 3.3: PATH, Launcher Where to Run: Code Directories What Not to Type: Prompts and Comments Running Code Interactively Why the Interactive Prompt? Usage Notes: The Interactive Prompt System Command Lines and Files A First Script Running Files with Command Lines Command-Line Usage Variations Usage Notes: Command Lines and Files Unix-Style Executable Scripts: #! Unix Script Basics The Unix env Lookup Trick The Python 3.3 Windows Launcher: #! Comes to Windows Clicking File Icons Icon-Click Basics Clicking Icons on Windows The input Trick on Windows Other Icon-Click Limitations vi | Table of Contents 43 44 45 46 47 48 49 50 52 54 55 56 57 58 59 59 60 60 62 62 63 63 66 Module Imports and Reloads Import and Reload Basics The Grander Module Story: Attributes Usage Notes: import and reload Using exec to Run Module Files The IDLE User Interface IDLE Startup Details IDLE Basic Usage IDLE Usability Features Advanced IDLE Tools Usage Notes: IDLE Other IDEs Other Launch Options Embedding Calls Frozen Binary Executables Text Editor Launch Options Still Other Launch Options Future Possibilities? Which Option Should I Use? Chapter Summary Test Your Knowledge: Quiz Test Your Knowledge: Answers Test Your Knowledge: Part I Exercises 66 66 68 71 72 73 74 75 76 77 78 79 81 81 82 82 82 83 83 85 85 86 87 Part II Types and Operations Introducing Python Object Types 93 The Python Conceptual Hierarchy Why Use Built-in Types? Python’s Core Data Types Numbers Strings Sequence Operations Immutability Type-Specific Methods Getting Help Other Ways to Code Strings Unicode Strings Pattern Matching Lists Sequence Operations Type-Specific Operations 93 94 95 97 99 99 101 102 104 105 106 108 109 109 109 Table of Contents | vii Bounds Checking Nesting Comprehensions Dictionaries Mapping Operations Nesting Revisited Missing Keys: if Tests Sorting Keys: for Loops Iteration and Optimization Tuples Why Tuples? Files Binary Bytes Files Unicode Text Files Other File-Like Tools Other Core Types How to Break Your Code’s Flexibility User-Defined Classes And Everything Else Chapter Summary Test Your Knowledge: Quiz Test Your Knowledge: Answers 110 110 111 113 114 115 116 118 120 121 122 122 123 124 126 126 128 129 130 130 131 131 Numeric Types 133 Numeric Type Basics Numeric Literals Built-in Numeric Tools Python Expression Operators Numbers in Action Variables and Basic Expressions Numeric Display Formats Comparisons: Normal and Chained Division: Classic, Floor, and True Integer Precision Complex Numbers Hex, Octal, Binary: Literals and Conversions Bitwise Operations Other Built-in Numeric Tools Other Numeric Types Decimal Type Fraction Type Sets Booleans viii | Table of Contents 133 134 136 136 141 141 143 144 146 150 151 151 153 155 157 157 160 163 171 Download from Wow! eBook pseudoprivate class attributes, 944–947 Python and, 16, 129, 554, 933 quiz questions and answers, 795 realistic example of classes, 817–857 state information, 1232 static and class methods, 1024–1034 super built-in function, 1041–1064 open built-in function customizing, 517–519 file processing and, 122, 126, 283 version considerations, 287 Windows platform and, 286 operations (see specific operations) operator module, 577 operator overloading about, 296, 785, 791, 805, 887, 1238 attributes and, 909–913, 1237–1256 binary operator methods, 917–921 Boolean tests and, 927–929 call expressions and, 921–925 common methods, 888–890 comparisons and, 925–927 constructors and expressions, 888 delegation and, 1322 display formats and, 368 double underscores and, 104 indexing and slicing, 890–894 iteration and, 894–906 membership and, 906–909 metaclasses and, 1374–1378, 1390 object destruction, 929–931 polymorphism and, 141 quiz questions and answers, 931 string representation and, 913–917 super built-in function and, 1047 usage considerations, 808 usage examples, 806–808, 826–828 validating methods, 1327 operator precedence, 139 optimizing objects about, 120, 1159 byte code files, 684 execution optimization tools, 37–38 optparse module, 754 or method, 889 or operator, 384 ord built-in function, 206, 582, 1167 OrderedDict subclass, 256 ordering (see sorting) 1528 | Index ORMs (object relational mappers), 13, 854 os module descriptor files, 295 _exit function, 1153 popen function, 295, 402, 411, 423, 607, 650, 1150 system function, 412, 1150 walk function, 423, 607 OSError class, 1133 OverflowError exception, 1131, 1132 P package imports about, 707–711 all variable, 711 from versus import statement, 713 init .py files, 709–711 relative imports model, 707, 717–733 search path settings, 708, 719 usage considerations, 713–716 usage example, 711–713 version considerations, 718 packages about, 707, 716 all variable, 711 namespace package model, 707, 723, 734– 741 package imports, 708–716 quiz questions and answers, 742 relative imports model, 707, 717–733 search path settings, 708, 719 parameters (see arguments) parentheses ( ) comprehensions and, 112 expression operators and, 139 statements and, 323, 328 superclasses and, 801 tuples and, 277 Parrot project, 40 parsing text in strings, 213 pass statement, 320, 389–390 passing-arguments-by-pointer, 524 passing-arguments-by-value, 524 PATH environment variable about, 1427, 1428 env program and, 60 new Windows options, 46 setting, 45, 57 paths module search paths, 72, 678–684, 723, 756–758, 1431, 1433 package imports, 708 package search paths, 708, 719 recording for recursive calls, 561 pattern matching in strings about, 108 re module and, 108, 215, 1206 pdb command-line debugger, 84, 1159 PEP (Python Enhancement Proposal), 15 percent sign (%) formatting expression operator, 217, 227– 229, 1189 system shell prompt, 44, 48, 56 performance considerations, 589 (see also benchmarking) list comprehensions, 589 MRO and, 1001 program execution, 32 Python alternatives, 35 slots, 1019 Perl programming language, 24 permutations, 612–614 persistence (see object persistence) Peters, Tim, 543 pexpect system, 295 pickle module about, 847 object persistence and, 13, 116 object serialization and, 1209–1211 persistence and, 941 storing objects, 290, 295 plus (+) operator adding numbers, 97 concatenating lists, 242, 1100 concatenating strings, 100, 200, 1100 PMW extension package, 11 polymorphism about, 101, 129, 933 classes and, 792–794, 832 in functions, 479, 482 OOP considerations, 934 operator overloading and, 141 testing exception types, 1151 portability, 17 positional arguments, 529, 534, 1331–1343 pow built-in function, 155 pprint module pformat function, 1006 pprint function, 1006 usage considerations, 1009 precedence rules, 139 print built-in function, 49, 359–361, 547–549 print operations, 49 (see also print statement) about, 358 built-in exception classes and, 1133–1135 completion certificate, 1414–1416 custom displays, 1135 display formats, 826–828 expression statements and, 356 file object methods and, 358 print built-in function, 49, 359–361, 547– 549 print stream redirection, 363–366 quiz questions and answers, 370 standard output stream, 295, 358, 368 version considerations, 359–363, 547–549, 821 version-neutral, 366–368 print statement about, 49, 361–363 common usage mistakes, 52, 59 debugging code and, 84 numeric display formats and, 143 private attributes, 1314–1318 procedures (see functions) profile module, 121, 642, 1158 program architecture about, 67 conceptual hierarchy, 93, 319 modules and, 670–673 program execution about, 27 alternative IDEs, 79–81 alternative launch options, 81–83 byte code compilation and, 30–31 clicking file icons, 62–66 debugging code, 83–85 development considerations, 32 embedding calls, 81 exec built-in function and, 72–73 frozen binaries, 39, 82 future possibilities, 40, 83 IDLE user interface, 73–79 interactive prompt, 31, 43–54 interpreters and, 27–28, 30 model variations in, 33–40 Index | 1529 module imports and reloads, 66–72 optimization tools, 37–38 performance considerations, 32 programmer's perspective, 28–30 PVM and, 31 quiz questions and answers, 41, 85–87 selecting from options, 83 system command lines and files, 54–59 text editor launch options, 82 Unix-style scripts, 59–62 program units, 96 (see also classes; functions; modules) Programming Python (Lutz), 985 programs about, 54 metaprograms, 759–761 modules and, 54, 93 prompts (see interactive prompt; system prompt) properties about, 1020–1023 attribute, 1221–1226 class statement, 798, 861, 1020 coding with decorators, 1224–1226 descriptors and, 1236–1237 validating with, 1256–1259 property built-in function, 1020, 1036, 1220, 1236 prototyping systems, 13 proxy classes (wrappers) about, 942–943 decorators installing, 1270 delegation and, 988 pseudoprivate class attributes about, 845, 944–947 larger projects and, 1160 public attributes and, 1321 PSF (Python Software Foundation), 15 pstats module, 1158 Psyco system, 36, 38 pth file extension, 708 public attributes, 1318–1321 PVM (Python Virtual Machine), 31 py file extension about, 29, 675, 687 common usage mistakes, 78 imported files and, 55 py2app tool, 39 py2exe tool, 39 1530 | Index pyc file extension, 30, 675, 676 pycache subdirectory, 31, 63, 675, 676– 678 PyChecker tool, 661, 1157 PyDev IDE, 80 PyDoc system about, 105, 444 changing colors in, 456 help function, 449–451, 883, 1157 HTML reports, 452–460 version considerations, 452–460 pydoc.py script, 458 pygame toolkit, 39 PyInstaller tool, 39, 1159 PyLint system, 1157 PyMongo interface, 116 pyo file extension, 684 PyPy system about, 7, 34 benchmarking, 652 performance considerations, 35 timeit module and, 643–645 Pyrolog interpreter, 36 PySerial extension, 295 PySolFC program, 15 pystone.py program, 656 Python Enhancement Proposal (PEP), 15 Python interpreter about, 27, 30 additional information, 1436 alternatives to, 36 byte code and, 30–31 configuring, 1427–1436 development considerations, 32 installing, 28, 1421–1425 locating with env program, 60 performance considerations, 32 PVM and, 31 Python programming language additional information, 1436 advantages of, 3–5 common applications of, 10–15 compared to other languages, 21–22 compared to Perl, 24 development community, 15 execution speed, future directions, 853–855 implementation alternatives, 33–36 new Windows options, 46 paradox of, 1409–1414 pillars of programming, 94 portability, 17 quiz questions and answers, 23–24 scripting and, technical strengths, 16–21 tools supporting, 10–15, 19, 1156 tradeoffs using, 8, 15 user base, 9–10, 35 version considerations (see version considerations for Python) Python Software Foundation (PSF), 15 Python Virtual Machine (PVM), 31 PYTHONIOENCODING environment variable, 1205 PythonLauncher, 62 PYTHONPATH environment variable about, 1427, 1428 module search paths, 72, 679, 756 package search paths, 708 PyDoc HTML reports, 458 Windows platform and, 1430 PYTHONSTARTUP environment variable, 1427, 1428 PythonWin IDE, 80 PyUnit tool, 1157 pyw file extension, 46 PY_PYTHON environment variable, 61, 1427, 1429 PY_PYTHON2 environment variable, 1427, 1429 PY_PYTHON3 environment variable, 1427, 1429 Q queue module, 496 queues best-first searches, 560 FIFO, 559 recursion versus, 559, 768 quiz questions and answers chapter 1: Python Q&A session, 23–24 chapter 2: program execution, 41 chapter 3: program execution, 85–87 chapter 4: object types, 131 chapter 5: numbers, 173 chapter 6: dynamic typing, 186 chapter 7: strings, 237 chapter 8: lists and dictionaries, 272 chapter 9: tuples, files, and everything else, 311 chapter 10: statements, 336 chapter 11: assignments, expressions, and prints, 370 chapter 12: if tests and syntax rules, 385 chapter 13: while and for loops, 414 chapter 14: iterations and comprehensions, 441 chapter 15: documentation, 466 chapter 16: functions, 483 chapter 17: scopes, 519–521 chapter 18: arguments, 551 chapter 19: functions, 578 chapter 20: comprehensions and generators, 626 chapter 21: benchmarking, 662 chapter 22: modules, 685 chapter 23: modules, 704 chapter 24: module packages, 742 chapter 25: modules, 777 chapter 26: OOP, 795 chapter 27: classes, 815 chapter 28: classes, 855–857 chapter 29: classes, 884 chapter 30: operator overloading, 931 chapter 31: classes, 978 chapter 32: classes, 1071 chapter 33: exceptions, 1090 chapter 34: exceptions, 1120 chapter 35: exception classes, 1139 chapter 36: exceptions, 1161 chapter 37: Unicode and byte strings, 1215– 1217 chapter 38: attributes, 1266–1268 chapter 39: decorators, 1344–1353 quotation marks interchangeable, 193 multiline block strings, 198–199 strings in, 105, 191 R r file processing mode, 122, 283 radd method, 889, 917–921 raise statement about, 321, 1081, 1106 built-in exceptions and, 1086 chaining exceptions, 1110–1112 from clause, 1110–1112 Index | 1531 propagating exceptions, 1110 raising exceptions, 549, 896, 1086, 1107 raising instances, 1126 signaling conditions with, 1147 version considerations, 1107 random module about, 98, 156 generator example, 616 range built-in function counter loops, 402 iteration and, 434, 435 list comprehensions and, 112, 242, 406 loop coding techniques and, 344, 402–407 nonexhaustive traversals, 405 sequence scans, 403 sequence shufflers, 404 timing calls example, 1298 rapid development cycle, 6, 33 rational numbers, 97, 127 raw_input built-in function, 64, 65, 330 re module about, 96 findall function, 1212 match function, 192, 1212 pattern matching and, 108, 215, 1206 search function, 1212 read-only descriptors, 1228 recursive comparisons, 300 recursive functions about, 487, 555, 880 coding alternatives, 556 from statement and, 775 generators and, 968 handling arbitrary structures, 558–561 loops versus, 557 reloaders, 764–767 summation with, 555 usage examples, 561 reduce built-in function, 430, 576 reference counters, 177, 179 references, 180 (see also shared references) about, 177, 500 assignments and, 339 attribute, 909 circular, 179 copies versus, 297–300, 308 cyclic, 179 dynamic typing and, 176, 180–185 1532 | Index string method calls, 231 weak, 185 relative imports model about, 707, 717–720 absolute imports versus, 722 lookup rules summary, 723 pitfalls of, 729–733 scope of, 722 usage considerations, 720–722 usage examples, 723–728 version considerations, 721 reload built-in function about, 66–71, 700–703 common usage mistakes, 71, 78 from statement and, 773–775 usage examples, 763–770 repetition as programming pillar, 94 in strings, 100, 200, 242 in tuples, 277 usage considerations, 309 repr built-in function about, 205 display formats, 144 string formatting method calls and, 224 version considerations, 138 repr method about, 889, 913–917 custom print displays, 1135 inheritance and, 842, 966 print display example, 826–828 recursive looping and, 561 reserved words, 352 reStructuredText markup language, 461 return statement about, 320, 477 coding functions, 475 function gotchas, 660 returning multiple values, 527 yield statement versus, 592–597 reversed built-in function, 248, 401 RIAs (rich Internet applications), 12 rich Internet applications (RIAs), 12 rmod method, 1189 round built-in function, 149, 156 running programs (see program execution) S SAX parsing, 1212 ScientificPython programming extension, 14 SciPy programming extension, 14, 111 scopes about, 485–488 accessing global variables, 498 builtins module, 491–493 comprehension variables and, 623 function-related gotchas, 661 global statement, 320, 476, 487, 494, 509 imports versus, 698 LEGB rule and, 488, 492, 872, 875–877 in methods and classes, 1068 minimizing cross-file changes, 497–498 minimizing global variables, 495 modules and, 695 name resolution and, 488–490 nested functions and, 499–508, 572 nonlocal statement, 320, 476, 487, 494, 508–517 quiz questions and answers, 519–521 relative imports model, 722 try/except statement and, 1108 usage example, 490 screen scraping technique, 854 scripts and scripting about, common usage mistakes, 78 error handling, 66 executable, 59–62 Internet, 11 launching scripts with icon clicks, 62–66 modules and, 54 Python support, 16 running with arguments, 1432 terminating compound statements, 53 timeit module and, 644, 647–651 timing script, 634 writing scripts, 55 search path modules, 72, 678–684, 756–758, 1431, 1433 packages, 708, 719 selection as programming pillar, 94 self argument about, 554, 863 coding constructors, 818 lambda callbacks and, 573 static methods and, 1025 usage considerations, 811 semicolon (;), 323, 327 sentinel value, 1147 sequence assignments about, 340–342 advanced patterns, 342–344 sequence operations bytes string type, 1190 dictionaries and, 258 generating scrambled sequences, 609–614 iteration and, 434 lists, 109 loop coding techniques, 403–405 numbers, 98 statement execution, 375 strings, 99–101 sequences about, 99 escape, 105, 191, 193–197 intersecting, 480–483, 545–547 list, 109, 243 as programming pillar, 94 repeating, 309 string, 99 tuple, 121 server connections, closing, 1148 set built-in function, 126, 167, 306 set comprehensions, 166, 168, 432, 624–626 set method about, 889 descriptors and, 912, 1293 managing attributes, 1227 set notation, 111 setattr built-in function, 572 setattr method about, 889, 909–913, 1220, 1238 attribute assignments and, 861 emulating privacy, 944 private attributes and, 1318 recursive looping and, 561 setitem method, 889, 890–894, 1194 sets about, 97, 126, 163, 169–171, 547 comparison operations, 302 copy method, 299 creating, 126 dictionary views and, 268 frozen, 167 immutable constraints, 167 literals, 96, 166 Index | 1533 version considerations, 164–169 setslice method, 893 shadowing, 661 shared references about, 180–181 arguments and, 524–526 augmented assignments and, 352 equality and, 183–185 in-place changes and, 181–183 multiple-target assignments and, 349 Shed Skin system, 33, 37 shell tools and commands, 6, 11, 411–413 (see also system command lines and files) shelve module about, 793, 847, 848 dictionary interfaces and, 271 exploring shelves interactively, 849–851 object persistence and, 116, 941 open function, 849 pickle module and, 1209–1211 storing objects on database, 848 updating objects, 851 shelves (see access-by-key databases and filesystems) singleton classes, 1301–1303 slice expressions, 891 slice objects, 891 slicing lists, 243, 244–246 nonexhaustive traversals, 405 operator overloading and, 890–893 strings, 100, 201–204 tuples, 277 slots about, 1008, 1010 descriptors and, 1237 example impacts of, 1017 handling generically, 1014–1016 managing attributes, 912 namespace dictionaries and, 1011–1013 private attributes and, 1318 speed considerations, 1019 superclasses and, 1013 usage rules, 1016 socket module, 96 sorted built-in function about, 118 dictionaries and, 269, 303 iteration and, 430 1534 | Index lists and, 248 tuples and, 278 sorting keys, 118–119, 256, 269 lists, 246–248 version considerations, 247, 302 source code about, 30 timestamps in, 31 spaces versus tabs, 378 Sphinx tool, 444, 461 SQL database API, 590 square brackets [ ], 96, 224, 328 square roots, 156 stack traces, 1084, 1099 Stackless Python, 11, 35 stacks inspecting, 1328–1329 LIFO, 249, 559 limiting depth of, 561 recursion versus, 559, 768 standard error stream (stderr), 366, 930 standard input stream (stdin), 369 standard library about, launch options, 82 standard output stream (stdout), 295, 358, 368 state information about, 129 built-in exception classes and, 1133–1135 class decorators and, 1317 in descriptors, 1232–1235 factory functions and, 501 function attributes and, 565 function decorators and, 1285–1289 generator functions, 592 nonlocal statement and, 512–517 recursive functions and, 560 validating with descriptors and, 1259–1263 statements about, 319 assignment, 320, 339–370 colon character and, 322 common usage mistakes, 53 compound (see compound statements) continuation lines in, 328, 378 control-flow, 375 delimiting, 378 expressions and, 93, 356–357 indenting, 324–327 interactive loops example, 329–336 listed, 320–322 modules and, 54, 93, 771 parentheses and, 323, 328 Python syntax model, 322–329 quiz questions and answers, 336 semicolon and, 323 special case rules, 327–329, 379–380 syntax rules, 375–380 terminating, 323 version considerations, 321 static methods about, 865, 1024 alternatives for, 1027 counting instances, 1030 usage considerations, 1028–1030 version considerations, 1025–1027 staticmethod built-in function, 951, 1024, 1029–1030 steps in slicing, 203 StopIteration exception, 417, 420, 593 storing objects and data binary data, 293 class building example, 847–853 class gotchas, 1069 in files, 288–290 in JSON format, 291–292 pickle module, 290, 295 on shelve database, 848 strings, 1170 struct module, 293 str built-in function about, 205, 306 display formats, 144 string formatting method calls and, 224 usage example, 98 version considerations, 138 str method about, 889, 913–917 custom print displays, 1135 inheritance and, 808, 966 print display example, 826–828, 843 str string type about, 1171–1173 converting, 1192 encoded text, 1183 re module and, 1206 text files and, 287 Unicode literals, 106, 190, 1176 version considerations, 194 stream redirection, 57 strides in slicing, 203 string formatting about, 103, 144, 1190 converting integers to strings, 152 expressions technique, 216–222 literals, 191 method calls technique, 222–234 nesting, 225 type codes, 218–220 string module about, 215–216 relative imports examples, 719–722, 724– 728 strings (str object) about, 96, 99, 189, 1167–1174 add method, 104 alternate ways to code, 105 backslash characters, 193–197 casefold method, 247 changing, 102, 208, 211–213 coding, 1174–1178 common operations, 190–201 comparison operations, 302 concatenating, 100, 104, 200, 1100 converting, 152, 192, 205–207 debugging, 972 decode method, 192, 1177 documentation, 199, 375, 444, 446–449 empty strings, 191 encode method, 192, 1169, 1177 endswith method, 192, 214 exceptions based on, 1124 find method, 102, 191, 200, 212 format method, 222, 225, 227–234, 968 formatting (see string formatting) garbage collection and, 208 immutable, 101, 191, 208 importing modules by name string, 761– 763 indexing, 99, 201–204 isdigit method, 192, 332 join method, 192, 213, 431, 598 literals, 96, 190–199, 378, 1175–1177 lower method, 192, 247 multiline block strings, 198–199 Index | 1535 nonexhaustive traversals, 405 operator overloading and, 913–917 parsing text, 213 pattern matching, 108, 215 quiz questions and answers, 237, 1215– 1217 repeating, 100, 200 replace method, 102, 191, 208, 211 rstrip method, 191, 205, 289, 426 sequence operations, 99–101 slicing, 100, 201–204 split method, 191, 213, 289 tool changes, 1206–1214 type and content mismatches, 1198 type-specific methods, 102, 191, 209–216 Unicode, 106–108, 189, 754–756, 1178– 1188 upper method, 103, 632 version considerations, 190, 194, 215–216, 229, 968, 1165–1167, 1204 strong typing, 97 struct module, 124, 293, 1172, 1207–1209 sub method, 888 subclasses about, 787 class interface techniques, 867–868 coding, 828 customizing behavior, 802, 828–834 extending built-in types, 981–983 inheritance and, 808, 866 type object and, 1366 subprocess module, 295, 413 substitution operations in string formatting, 103 sum built-in function, 112, 155, 555 super built-in function about, 831, 865, 1041 basic usage and tradeoffs, 1043–1049 debates about, 1041–1042 multiple inheritance and, 1043–1046, 1050–1062 operator overloading and, 1047 runtime class changes, 1049 summary of, 1062 version considerations, 1048 superclasses about, 787 abstract, 869–871, 939 class gotchas, 1069 1536 | Index class interface techniques, 867–868 constructor methods, 864 customizing, 833 inheritance and, 866 metaclasses versus, 1381 multiple inheritance and, 956 operator overloading methods and, 1325 parentheses and, 801 slots and, 1013 traditional forms, 1042 sys module argv attribute, 204, 650, 751 excepthook function, 1149 exc_info function, 1138, 1149, 1150–1152 exit function, 1146, 1153 file name settings, 1205 getrecursionlimit function, 561 modules dictionary, 498, 676, 759 path list, 72, 682, 719, 723, 756 platform attribute, 55, 223, 631 setrecursionlimit function, 561 stderr attribute, 366, 930 stdin attribute, 369 stdout attribute, 295, 358, 363–366, 368 system command lines and files about, 54, 1432–1436 common usage mistakes, 58 running files with command lines, 56 running in Python, 294, 411–413, 423, 650, 770 starting interactive sessions, 44 system shell prompt, 44, 48 timeit module and, 644 usage variations, 57 writing scripts, 55 system shell prompt about, 44, 48 running files, 56 SystemExit exception, 1146, 1153 systeminfo command, 412 systems programming, 11 T -t command-line flag, 378 tabs versus spaces, 378 TCL_LIBRARY environment variable, 1427 termination actions about, 1083, 1087–1088 default exception handler and, 1084 Download from Wow! eBook try/finally statement and, 1083, 1087–1088, 1100–1102, 1152 with/as statement and, 1083, 1088, 1152 testing error handling, 332–334 filtering results, 427 from statement and, 774 interactive prompt and, 51 list comprehensions and, 583–586 for missing keys, 117 with name attribute, 750 for positional arguments, 1331–1343 processes, 819–820, 1149 reloading variants, 769 timing calls example, 1297 truth values, 171, 305, 380–382, 927–929 type, 986, 992–995, 1342 text editor launch options, 82 text files about, 107, 1173–1174, 1196 binary files and, 123 creating, 122 Unicode, 124–126, 190, 754–756 version considerations, 287, 1197 _thread module, 496 threading module, 496 3to2 converter, 367 time module about, 121, 543 clock function, 631, 633, 644, 1297 homegrown timing module and, 630 perf_counter function, 633, 1297 process_time function, 633 time function, 631, 633 timeit module about, 121, 543, 642–647 benchmark and script, 647–651 other examples, 890, 1019, 1297, 1434, 1484 repeat function, 643 setup code, 646, 654 timestamps in source code, 31 timing calls with function decorators, 1295– 1301 timing iterations alternatives for, 629–642 timeit module, 642–655 timsort algorithm, 543 tkinter GUI toolkit about, 11 callbacks and, 923, 953 common usage mistakes, 79 configuring Python and, 1428 IDLE and, 74 keyword arguments and, 550 lambda callbacks, 573 quit function, 79 testing reloading variants, 769 TK_LIBRARY environment variable, 1427 traceback module, 1151 traceback objects, 1150, 1151 translation (see encoding and decoding) triple quotes, 198–199 true value in Python Booleans and, 171, 304–305, 380–382 built-in scope and, 494 operator overloading and, 927–929 truncating division, 146–150 try statement about, 1094–1095 catching built-in exceptions, 1100 clauses supported, 1095–1098 debugging with, 1149 default behavior, 1098 wrapping statements with, 1152 try/except statement about, 321, 1081 catching exceptions, 1084, 1088, 1096– 1097, 1100 error handling, 333–334 nesting, 1141–1145 scopes and, 1108 try/except/else statement, 1093–1100, 1147, 1150 try/except/finally statement, 1102–1106, 1143–1145 try/finally statement about, 321, 1081 closing files and server connections, 1148 closing files example, 294 termination actions, 1083, 1087–1088, 1100–1102, 1152 -tt command-line flag, 378 tuple built-in function, 279, 306, 431 tuple-unpacking assignments, 340, 396–398 tuples about, 70, 121–122, 276 assignments and, 591 Index | 1537 common operations, 276, 277–279 comparison operations, 302 concatenating, 277 converting, 278–279 count method, 277, 279 empty, 276 exception hierarchies and, 1129 immutable, 121, 278–279 index method, 277, 279 indexing, 277 iteration in, 277 lists versus, 279 literals, 96, 276 named, 122, 256, 277, 280–282 nesting, 277 quiz questions and answers, 311 repeating, 277 slicing, 277 tuple keys, 259 type-specific methods, 278–279 2to3 converter, 366 type built-in function, 128, 306, 986, 992–995 type designators, 177 type object classes as instances of, 1364–1366 inheritance and, 1379 metaclasses as subclasses of, 1366 TypeError exception, 1100, 1356 types module, 306, 764 U unbound methods, 948–953, 1025 underscore (_) class names, 845 module names, 70, 747 name mangling and, 945 operator overloading, 104, 805 showing name values, 971 Unicode character set character code conversions, 207 code points, 106, 194, 206, 1170 currency symbols, 754–756 encoding and decoding, 123, 190, 192, 1178–1188, 1199–1206 JSON format and, 292 literals and, 106, 190, 1176 quiz questions and answers, 1215–1217 strings and, 106–108, 189, 754–756, 1178– 1188 1538 | Index text files and, 124–126, 190, 754–756 unicode string type about, 106, 190, 287, 1171–1173 coding strings with, 1185–1187 converting, 1192 re module and, 1206 unittest module, 750, 1158 Unix platform awk utility, 413 configuring Python, 1430 env program, 60 executable scripts, 59–62 frozen binaries, 39 GUI support, 11 icon clicks, 62 IDLE startup details, 75 installing Python, 28, 1425 system shell prompt, 44 Windows launcher and, 1437 working directory, 48 Unladen Swallow project, 40 unpacking arguments, 528, 535 user-defined classes, 129 user-defined exceptions, 1086, 1134, 1147 UTF-16 encoding, 1170 UTF-8 encoding, 1168 V validating attributes, 1256–1266 function arguments, 1330–1343 operator overloading methods, 1327 value equality operators, 137 van Rossum, Guido, 17 varargs, 529, 536–537 variables about, 177 assigning values to, 50, 99, 176, 177 attributes and, 68 comprehension, 490, 623 creating, 99, 176 dynamic typing and, 176–178 exception, 490 expressions and, 176 from * statement and, 773 function-related gotchas, 661 global, 495, 498, 745 local, 483 name collisions and, 71 name rules for, 352–355 numbers in, 141–143 objects and, 176 scope of, 486 shared references and, 180–185 try/except statement and, 1108 version considerations for Python about, xxxvi–xxxix abstract superclasses, 870 Booleans, 928 builtins module, 156, 493 classes, 983 comparisons and sorts, 247, 302 context managers, 1118–1119 dictionaries, 264–271, 303 division operations, 146–148 exception classes, 1123 expression operators, 138 files, 287, 1197 function attributes, 515–517 iteration, 419, 434–440, 896 keyword arguments, 539–542 map built-in function, 408 metaclasses, 1369–1370 next method, 593 nonlocal statement, 508–512 package imports, 718 printing, 359–363, 547–549, 821 PyDoc system, 452–460 raise statement, 1107 relative imports model, 721 sets, 164–169 statements, 321 static methods, 1025–1027 storing strings in memory, 1170 strings, 190, 194, 215–216, 229, 968, 1165– 1167, 1204 summarized, 1451–1463 super built-in function, 1048 threading, 496 unbound methods, 950 wrapper classes, 943 view objects, 266–269, 439–440 virtual attributes, 1014–1016 virtual concatenation, 737 W w file processing mode, 122, 283 warnings module, 1147 weak references, 185 weakref module, 185 websites, 462, 853 while statement about, 119, 320, 387 C language, 394 general format, 388 interactive loops example, 330 iteration and, 418 quiz questions and answers, 414 recursion versus, 557 sequence scans, 403 usage examples, 388 whitespace, 191 win32all package, 716 Windows launcher #! comment support, 60–62 about, 46, 57, 1439–1441, 1450 command lines, 1435 icon clicks, 62 pitfalls, 1447–1450 tutorial on, 1441–1447 Unix legacy, 1437 Windows legacy, 1438 Windows platform #! comment support, 60–62 command-line interface, 45, 57 common usage mistakes, 58 configuring Python, 1430 frozen binaries and, 39 GUI support, 11 icon clicks, 62, 63–65 IDLE startup details, 74 installing Python, 28, 1424, 1425–1427 new options, 46 open built-in function, 286 Python documentation, 461 system shell prompt, 44 systeminfo command, 412 win32all package, 716 working directory, 47 Windows Registry Editor, 1431 Wing IDE, 80 Winpdb system, 85 with/as statement about, 321, 1081, 1114–1117 closing files and server connections, 1148 file objects and, 285, 294 resetting precision, 159 Index | 1539 termination actions, 1083, 1088, 1152 version considerations, 1118–1119 working directory, 47 wrappers (proxy classes) about, 942–943 decorators installing, 1270 delegation and, 988 writing scripts, 55 wxPython GUI API, 11 X _x naming convention, 747, 1321 XML parsing tools, 1211–1214 xrange built-in function, 403, 435 Y yield operator, 137 yield statement about, 320 coding example, 902–906 coding functions, 475 extended syntax, 605 function gotchas, 660 generator functions and, 591 iteration and, 423, 440 return statement versus, 592–597 Z ZeroDivisionError exception, 1131, 1132 zip built-in function dictionary keys and, 262, 265 iteration and, 430, 433, 434, 437, 617–621 loop coding techniques and, 402, 407–410 parallel traversals, 407–410 zip file extension, 684 ZODB object-oriented database system, 854 1540 | Index About the Author Mark Lutz is a leading Python trainer, the author of Python’s earliest and best-selling texts, and a pioneering figure in the Python world Mark is the author of the three O’Reilly books Learning Python, Programming Python, and Python Pocket Reference, all currently in fourth or fifth editions He has been using and promoting Python since 1992, started writing Python books in 1995, and began teaching Python classes in 1997 As of Spring 2013, Mark has instructed 260 Python training sessions, taught roughly 4,000 students in live classes, and written Python books that have sold 400,000 units and been translated to at least a dozen languages Together, his two decades of Python efforts have helped to establish it as one of the most widely used programming languages in the world today In addition, Mark has been in the software field for 30 years He holds BS and MS degrees in computer science from the University of Wisconsin where he explored implementations of the Prolog language, and over his career has worked as a professional software developer on compilers, programming tools, scripting applications, and assorted client/server systems Mark maintains a training website and an additional book support site on the Web Colophon The animal on the cover of Learning Python, Fifth Edition, is a wood rat (Neotoma Muridae) The wood rat lives in a wide range of conditions (mostly rocky, scrub, and desert areas) over much of North and Central America, generally at some distance from humans Wood rats are good climbers, nesting in trees or bushes up to six meters off the ground; some species burrow underground or in rock crevices or inhabit other species’ abandoned holes These grayish-beige, medium-size rodents are the original pack rats: they carry anything and everything into their homes, whether or not it’s needed, and are especially attracted to shiny objects such as tin cans, glass, and silverware The cover image is a 19th-century engraving from Cuvier’s Animals The cover font is Adobe ITC Garamond The text font is Linotype Birka; the heading font is Adobe Myriad Condensed; and the code font is LucasFont’s TheSansMonoCondensed Download from Wow! eBook ... FIFTH EDITION Download from Wow! eBook Learning Python Mark Lutz Beijing • Cambridge • Farnham • Kưln • Sebastopol • Tokyo Learning Python, Fifth Edition by Mark Lutz Copyright... the O’Reilly logo are registered trademarks of O’Reilly Media, Inc Learning Python, 5th Edition, the image of a wood rat, and related trade dress are trademarks of O’Reilly Media, Inc Many of... Keuren June 2013: Indexer: Lucie Haskins Cover Designer: Randy Comer Interior Designer: David Futato Illustrator: Rebecca Demarest Fifth Edition Revision History for the Fifth Edition: 2013- 06-07