Learning Python FOURTH EDITION Learning Python Mark Lutz Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo Learning Python, Fourth Edition by Mark Lutz Copyright © 2009 Mark Lutz All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://my.safaribooksonline.com) For more information, contact our corporate/institutional sales department: (800) 998-9938 or corporate@oreilly.com Editor: Julie Steele Production Editor: Sumita Mukherji Copyeditor: Rachel Head Production Services: Newgen North America Indexer: John Bickelhaupt Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Robert Romano Printing History: March 1999: December 2003: October 2007: September 2009: First Edition Second Edition Third Edition Fourth Edition Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc Learning Python, the image of a wood rat, and related trade dress are trademarks of O’Reilly Media, Inc Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein ISBN: 978-0-596-15806-4 [M] 1252944666 To Vera You are my life Table of Contents Preface xxxi Part I Getting Started A Python Q&A Session Why Do People Use Python? Software Quality Developer Productivity Is Python a “Scripting Language”? OK, but What’s the Downside? Who Uses Python Today? What Can I Do with Python? Systems Programming GUIs Internet Scripting Component Integration Database Programming Rapid Prototyping Numeric and Scientific Programming Gaming, Images, Serial Ports, XML, Robots, and More How Is Python Supported? What Are Python’s Technical Strengths? It’s Object-Oriented It’s Free It’s Portable It’s Powerful It’s Mixable It’s Easy to Use It’s Easy to Learn It’s Named After Monty Python How Does Python Stack Up to Language X? 5 7 9 10 10 11 11 11 12 12 13 13 13 14 15 16 16 17 17 17 vii Chapter Summary Test Your Knowledge: Quiz Test Your Knowledge: Answers 18 19 19 How Python Runs Programs 23 Introducing the Python Interpreter Program Execution The Programmer’s View Python’s View Execution Model Variations Python Implementation Alternatives Execution Optimization Tools Frozen Binaries Other Execution Options Future Possibilities? Chapter Summary Test Your Knowledge: Quiz Test Your Knowledge: Answers 23 24 24 26 29 29 30 32 33 33 34 34 34 How You Run Programs 35 The Interactive Prompt Running Code Interactively Why the Interactive Prompt? Using the Interactive Prompt System Command Lines and Files A First Script Running Files with Command Lines Using Command Lines and Files Unix Executable Scripts (#!) Clicking File Icons Clicking Icons on Windows The input Trick Other Icon-Click Limitations Module Imports and Reloads The Grander Module Story: Attributes import and reload Usage Notes Using exec to Run Module Files The IDLE User Interface IDLE Basics Using IDLE Advanced IDLE Tools Other IDEs Other Launch Options viii | Table of Contents 35 37 38 39 41 42 43 44 46 47 47 49 50 51 53 56 57 58 58 60 62 63 64 in-place change operations, 388 incremental prototyping, 645 indentation, 266–269, 314, 388 rules, 315 tabs versus spaces, 317 index method, 206 and tuples, 228 indexing, 165, 166 indexing expressions, 80 indirect function calls, 469 infinite loops, 328 inheritance, 612, 613, 629–632, 687–693 abstract superclasses, 690–693 attribute inheritance, key ideas of, 629 attribute trees, 687 class interface techniques, 689 real-world relationships, modeling with, 739 simplicity of inheritance model, 636 specializing inherited methods, 687 input function, 49 insert method, 87, 206 installing Python, 23 instance methods, 800 instances, 614, 615, 625, 626, 643 making instances, 644–648 coding constructors, 644 incremental testing, 645 test code, 646 as namespace objects, 638 int, 169 int function, 235 integer division, Python 2.6 versus 3.0, 115 integers, 106 Python 3.0, 107 integrated development environments (see IDEs) interactive loops, 271–276 math operations on user input, 272 nesting code three levels deep, 275 simple example, 271 testing inputs, 273 try statements, handling errors with, 274 interactive prompt, 35–41 exiting a session, 37 experimenting with code, 38 files, running from, 43 multiline statements, entering, 41 testing code, 39 1148 | Index tips for using, 39 Internet scripting, 10 interpreters, 23 introspection, 591 introspection attributes, 1053 IronPython, 30, 1091 is operator, 244 “is-a” relationships, 739 is_integer method, 108 items method, 211, 370 iter function, 354 iteration, 485 built-in tools for, 362 manual iteration, 354 iteration protocol, 94, 351, 352, 493 iterators, 351–358 additional built-in iterators, 356 file iterators, 352 filter, 368 generator functions (see generator functions) map, 368 in Python 3.0, 366–371 range, 367 support for multiple iterators, 369 range function, 342 timing iteration alternatives, 509–518 other suggestions, 517 time module, 509 time module alternatives, 513 timing results, 511 timing script, 510 zip, 368 iters.py, 712 J JIT (just-in-time) compilation, 31 jump tables, 476 Jython, xlv, 29, 1091 K keys, 93 keys method, 370 keyword arguments, 204, 460, 646 keyword-only arguments (Python 3.0), 450 justification, 453 ordering rules, 452 Komodo, 63 L lambda expression, 474–479 basics, 474 defining inline callback functions in tkinter, 479 justification for, 475 nested lambdas and scopes, 478 potential for code obfuscation, 477 lambdas and nested scopes, 422 Latin-1 character encoding, 897 LEGB rule, 410 len function, 80 lexical scoping, 408 Linux Python command line, starting, 36 list comprehension expressions, 88 list comprehensions, 351, 358–362, 485 basics, 359 best uses of, 490 extended syntax, 361 files, using on, 359 map function and, 491 map function, versus, 486 matrixes and, 489 tests and nested loops, adding, 487 list object type, 86–90 bounds checking, 87 nesting, 88 type specific operations, 87 list-unpacking assignment statements, 280 lister.py, 758 ListInstance class, 758–761 lists, 197 basic operations, 200 changing in place, 202–207 deleting items or sections in place, 206 index and slice assignments, 202 list method calls, 203–206 coding of lists, 199 common literals and operations, 198 indexing, slicing, and matrixes, 201 iteration and comprehensions, 200 literals, 77 local scope, 409 local variables, 404 long integers (Python 2.6), 107 loop else block, 329 loop statement versus recursive functions, 467 loops, 327–349 break, continue, pass, and loop else clause, 329 coding techniques, 341–349 counter loops with while and range, 342 for statement, 334–341 general format, 329 generation of offsets and items, 348 interactive loops (see interactive loops) loop else clause, 332 nonexhaustive traversals with range and slices, 343 parallel traversals with zip and map, 345 range function lists, changing with, 344 while statement, 327 M Mac OS X Python command line, starting, 36 makedb.py, 671 managed attributes, 941–947 attribute validations example, 973–979 descriptors validation, 975 properties validation, 973 getattribute , validation with, 978 getattr , validation with, 977 coding to run on attribute access, 942 comparison of management techniques, 963 descriptors, 947–956 computed attributes, 952 descriptor methods, 948 example, 950 method arguments, 948 properties, relation to, 955 read-only descriptors, 949 state information, using in, 953 justification, 941–943 properties, 943–947 computed attributes, 945 decorators, coding with, 946 first example, 944 new-style object derivation requirement, 944 getattr and getattribute , 956–973 avoiding loops, 958 comparison, 962 computed attributes, 961 delegation, 958 example, 959 Index | 1149 interception of built-in attributes, 966– 973 Manager class, 643, 653 manager functions, 1054 manual iteration, 354 manynames.py, 694 map function, 201, 346, 479 lambda expressions and, 480 list comprehensions and, 491 list comprehensions, versus, 486 map iterator, 368 matching algorithm, 1042 math module, 79 example functions, 125 mathematical functions, 108 max and functions, 364 mergdexc.py, 846 metaclass model, 1058–1062, 1058 (see also metaclasses) class statement protocol, 1061 classes are instances of type, 1058 metaclasses are subclasses of type, 1061 metaclasses, 781, 794, 807, 1051–1056, 1058 (see also metaclass model) adding methods to classes example, 1070– 1076 manual augmentation, 1070 metaclass-based augmentation, 1071 applying decorators to methods example, 1076–1083 manual tracing, 1076 tracing with metaclasses and decorators, 1077 with any decorators, 1079 class decorators, compared to, 1073–1076, 1080 decorator-based augmentation, 1073 managing instances instead of classes, 1074 class decorators, compared with, 1056 coding, 1063–1069 basic metaclass, 1064 customizing construction and initialization, 1065 factory functions, using, 1066 instances versus inheritance, 1068 overloading class creation calls with classes, 1067 1150 | Index overloading class creation calls with metaclasses, 1066 declaration, 1062 issues around use, 1052 potential roles, 1052 metafunctions, 804 metaprograms, 591–594 method call expression, 173 methods, 82, 84, 203, 638, 650, 684–686 augmenting methods, 654 bound and unbound methods, 750–756 bound methods, 728 calls to methods, 685–686 class methods, 686 coding methods, 649 comparison methods, 728 destructor method, 732 example, 685 static methods, 686, 795 (see also static methods) superclass constructors, calling, 686 and max functions, 125 mins.py, 454 mix-in classes, 757 coding, 757–767 instance attributes, listing, 758 listing inherited attributes, 761 listing object attributes in class trees, 763 module packages, 561 package imports, 561–569 import example, 564–566 justification, 566–569 packages and search path settings, 562 inti .py, 563 package relative imports (see package relative imports) modules, 42, 51, 75, 529 as extension for import and from, 591 attributes, 53, 531, 543 classes, as attributes of, 631 classes, versus, 703 creating, 543 data hiding in, 583 design concepts, 598 exec, running module files with, 57 from statement, 545 future language features, enabling, 584 global scope, 408 gotchas, 599–604 from imports and reload, 601 from statement, 600 from statement and variables, 601 recursive import failures, 603 statement order in top-level code, 599 import statement, 544 importing byte code compilation if required, 534 running, 535 importing by name string, 594 importing of modules, 533–535 locating, 534 imports and reloads, 51–57 metaprograms, 591–594 mixed-usage modes, 585–589 module extensions, 544 module namespaces, 550–554 attribute name qualification, 552 generation from files, 550 imports versus scopes, 552 namespace nesting, 553 module search path, 56, 534, 535–541 advanced module selection concepts, 540 module file selection, 539 search path configuration, 537 search path variations, 538 sys.path list, 538 third-party extensions, 540 module search path, changing, 590 namespaces, 55, 529 naming conventions, 644 naming of, 543 program structure and, 530 reloading modules, 554–558 example, 556 roles of, 530 scope, 550 standard library, 533 transitive module reloads, 595–598 name attribute, 585–589 command-line arguments with, 587 unit tests, 586 Monty Python’s Flying Circus, 17 multiline statements, 317 multiple inheritance, 617, 756–767 diamond pattern inheritance trees, 783 mix-in classes (see mix-in classes) multiway branching, 312 mutable objects, 197, 649 mutables in assignments, 388 mybooks.xml, 934 mydir.py, 592 myfile.py, 53 N name attribute, 102 name mangling, 747–750 namespace objects, 646 namespaces, 53, 55, 407, 529, 615, 693–701 attribute names, 693 name assignment, 694–696 namespace dictionaries, 696–699 namespace links, 699 simple names, 693 negative offsets, 165 nested scopes, 419–425 abitrary nesting, 424 examples, 419 factory functions, 420 lambdas and, 422 nester.py, 812 NET and IronPython, 30 NetBeans, 63 “new-style” classes, 777–794 changes, 778–787 class extensions, 788–795 class properties, 792–794 instance slots, 788–791 metaclasses, 794 multiple slot lists in superclasses, 790 slots and generic code, 788 getattribute method, 794 diamond inheritance change, 783–787 example, 784 explicit conflict resolution, 785 scope of search order, 787 type model changes, 779–783 object type objects, 782 type testing implications, 781 next function, 354 next method, 352 non-ASCII text, coding, encoding and decoding, 905–906 None object, 247 Index | 1151 nonlocal statement, 409, 425–432 absence from Python 2.6, 263 basics, 425 examples, 426 boundary cases, 428 justification for, 429–432 Python 2.6 alternatives, 429 normal integers (Python 2.6), 107 number object type, 78 number operations, 113–127 bitwise operations, 124 comparisons, 116 chained comparisons, 116 complex numbers, 122 division, 117–121 integer precision, 121 math module functions, 125 notation, hexadecimal, octal, and binary, 122 numeric display formats, 115 variables and basic expressions, 113–115 numeric display formats, 115 numeric extensions, 140 numeric object type, 105–108 built-in numeric tools, 108 complexity ranking, 112 expression operators and precedence, 108 numeric literals, 106 operator overloading and polymorphism, 112 some noncore types, 127–140 Booleans, 139 decimal type, 127–129 fraction types, 129–133 sets, 133–139 numeric precision, setting globally, 128 numeric programming, 11 NumPy numeric programming extension, 7, 11, 140 O object embedding, 661 object persistence, 669 object type categories, 193 mutable versus immutable types, 194 shared operation sets, 194 object types, 102 built-in object types (see built-in object types) 1152 | Index object-oriented programming, 13 object-oriented scripting language, object.attr expression, 687 object.attribute expression, 613 object.attribute notation, 532 objects, 75, 105, 145 comparisons, equality, and truth, 244 compound object types, 241 copying versus referencing, 150 databases, storing objects in, 669 dynamic typing and, 144 iterable objects, 352 nonbuilt-in object types, 250 object classifications for built-in types, 240 references versus copies, 241 truth and falsity, 246 bool type, 248 None object, 247 type hierarchies, 248 type object type, 250 octal numeric literals, 107 OOP (object-oriented programming), 613– 621 as exemplified by coding of classes, 658 attribute inheritance search, 613 class method calls, 616 classes, 614, 615, 625 class trees, 616–619 customization by inheritance, 629–632 code reuse, 619–621 design issues, 737 bound and unbound methods, 750–756 composition, 740–745 delegation and wrapper classes, 745 generic object factories, 768 inheritance, 739 multiple inheritance, 756–767 polymorphism, 738 pseudoprivate class attributes, 747–750 design patterns, 621 important concepts, 660 instances, 614, 615, 625 object.attribute expression, 613 open call, (Python 2.6), 912 open function, 97, 99, 230 mode string argument, 901 operator overloading, 240, 613, 633–636, 651– 653, 652, 705–733 attribute references, 718–720 attribute privacy, 720 Boolean tests, 730 call expressions, 725–728 function interfaces, 727 common operator overloading methods, 706 comparisons, 728 constructors and expressions, 706 index iteration, 710 indexing and slicing, 708–710 Python 2.6, 709 iterator objects, 711 multiple iterators on one object, 714 user defined iterators, 712 justification, 636 membership, 716 comparisons, Python 2.6, 716 object destruction, 732 overloading methods, 1053 overloading methods in Python 2.6, 747 overview, 705–707 right-side and in-place addition, 723–725 string representation, 721–723 operator precedence, 111 optimization, 889 ORMs (object-relational mappers), 676 OverflowError class, 865 P package imports, 561 justification, 566–569 import versus from, 569 root directory, 568 packages and search path settings, 562 package relative imports, 569–581 absolute package paths, versus, 573 basics, 570 examples, 575–581 imports outside packages, 575 imports relative to CWD, 577 imports still relative to CWD, 579 imports within packages, 576 modules, selecting with relative and absolute imports, 578 justification, 572 module lookup rules summary, 575 Python 3.0, 572 changes, 570 versus 2.6, 570 scope, 574 packages, 561 parallel traversals, 345 parentheses ( ), 265, 269, 318 functions and, 389 generator expressions and, 497 tuples and, 96 Parrot project, 33 parsing, 165 pass statement, 329, 330 PATH environment variable, 36, 1093 pattern matching, 85 pdb debugger, 889 PEP (Python Enhancement Proposal) protocol, 12 Person class, 644 incremental testing, 645 subclassing, 653 version portability, 647 person.py, 644 Peters, Tim, 1052 pickle module, 236, 670, 744 binary data requirement, 933 string serialization (Python 3.0), 932 pizzashop.py, 741 polymorphism, 82, 101, 113, 401, 403, 620 dynamic typing and, 153 example, 656 overloading in Python versus other languages, 738 pop method, 87, 205, 211 portability, 14 Portable Python, 1090 pow function, 125 precedence parentheses and, 111 precedence rules, 111 print function, 263 print operations, 297–307 print and stdout, 307 print function (Python 3.0), 298 print statement (Python 2.6), 300 print stream redirection, 302 version-neutral printing, 306 print statement (Python 2.6), 263 procedure, 396 profile module, 517, 888 profilers, 888 program execution, 24–28 Index | 1153 development implications, 28 Python compared to other languages, 27 program portability, program shipping options, 889 program structure, imports, 531 program units, 77 programs, 5, 42, 75 icons, opening with, 47 launching, 35 additional launch options, 64 choosing a launch option, 66 clicking file icons, 47 exec function, 57 from the command line, 43 IDLE user interface, 58 input function, 49 module imports and reloads, 51, 56 Unix executable scripts, 46 Windows automatic file extensions, 44 running interactively, 35–41 experimentation, 38 testing, 39 saving in files, 41 structure, 261, 530–533 Windows, saving under, 45 property built-in function, 942 computed attributes, 945 decorators, 946 first example, 944 proxy classes, 745 pseudoprivate attributes, justification, 748 pseudoprivate names, 747 PSF (Python Software Foundation), 13 Psyco just-in-time compiler, 30, 889 pth file extension, 1096 pth path file directories, 537 PVM (Python Virtual Machine), 27 py file extension, 25, 43, 543 py2exe, 32, 889 pyc file extension, 26, 534 PyChecker, PyLint, and PyUnit, 887 PyDev, 63 PyDoc, 84, 380–385, 887 help function, 380 HTML reports, 383 PyInstaller, 32, 889 PyPy project, 33 Python command-line options, 1097–1098 1154 | Index configuration, 1093 DOS variables in autoexec.bat, 1095 environment variables, 1093 path files, 1096 setting configuration options, 1095 Unix/Linux shell variables, 1095 Windows environment variable GUI, 1095 Windows registry, 1096 installing, 23 Python 2.0 string module, 178 Python 2.6, xxxii backward compatibility to older versions, xxxii binary and Unicode strings, handling of, 895 Booleans, 731 iteration method X.next( ), 356 nonlocal statement, alternatives to in, 429 operator overloading methods, 747 Python 3.0 print function, emulating, 457 using keyword-only arguments, 459 raw_input function, 50 reload function, 53 string object types, 157 unicode and str operation sets, 911 “new-style” and “classic” classes, 777 Python 3.0, xxxii built-in attributes, 662 compared 2.x versions, xxxv comparisons and sorts, 204 dictionary changes, 217 dictionary comparisons, 246 extended sequence unpacking, 281, 284 in for loops, 338 function annotations, 472 incompatibility with older versions, xxxii input function, 50 new iterables, 366–371 “new-style” classes, 777 older version tools removed from, xxxvi special character coding, 908 string object types, 157 string type, changes in, 896 text and binary files, 233 Unicode and binary data support, 895 unsupported raise syntax, 850 Python interpreter, 23 installing, 1089 on Linux, 1091 on PDAs, 1092 on Unix, 1091 on Windows, 1091 on Windows Vista, 1092 website downloads link, 1090 Python programming language, xxxi advantages of, 3–5 common coding gotchas, 387 compared to other languages, 17 developer productivity and, development tools, 886 documentation (see documentation) execution speed, iteration protocol, 493 manuals and resources, 1098 old and new versions, xxxii Perl, compared to, 20 portability, 14 primary implementations of, 29 so-called “optional” features, 1082 statically-typed languages, compared to, 401 string model, 897–902 support, 12 technical strengths, 13–17 user base, uses for, 9–12 PYTHONPATH, 1094 PYTHONPATH directories, 536 PYTHONSTARTUP, 1094 PythonWin, 63, 1090 PyWin32, 1090 Q quizzes, xliii Chapter 1: A Python Q&A Session, 19 Chapter 2: How Python Runs Programs, 34 Chapter 3: How You Run Programs, 68 Chapter 4: Introducing Python Object Types, 103 Chapter 5: Numeric Types, 141 Chapter 6: The Dynamic Typing Interlude, 153 Chapter 7: Strings, 195 Chapter 8: Lists and Dictionaries, 223 Chapter 9: Tuples, Files, and Everything Else, 253 Chapter 10: Introducing Python Statements, 276 Chapter 11: Assignments, Expressions, and Prints, 308 Chapter 12: if Tests and Syntax Rules, 324 Chapter 13: while and for Loops, 349 Chapter 14: Iterations and Comprehensions, Part 1, 372 Chapter 15: The Documentation Interlude, 389 Chapter 16: Function Basics, 405 Chapter 17: Scopes, 432 Chapter 18: Arguments, 461 Chapter 19: Advanced Function Topics, 483 Chapter 20: Iterations and Comprehensions, Part 2, 523 Chapter 21: Modules, the Big Picture, 541 Chapter 22: Module Coding Basics, 558 Chapter 23: Module Packages, 582 Chapter 24: Advanced Module Topics, 604 Chapter 25: OOP: The Big Picture, 622 Chapter 26: Class Coding Basics, 641 Chapter 27: A More Realistic Example, 677 Chapter 28: Class Coding Details, 703 Chapter 29: Operator Overloading, 734 Chapter 30: Designing with Classes, 770 Chapter 31: Advanced Class Topics, 815 Chapter 32: Exception Basics, 833 Chapter 33: Exception Coding Details, 856 Chapter 34: Exception Objects, 870 Chapter 35: Designing with Exceptions, 891 Chapter 36: Unicode and Byte Strings, 937 Chapter 37: Managed Attributes, 979 Chapter 38: Decorators, 1047 Chapter 39: Metaclasses, 1084 quotes escaping, 158 strings and, 158 triple quotes, 162 R raise statement, 829, 848–850 from clause Python 3.0 exception chaining, 849 Index | 1155 nonerror conditions, signaling with, 878 random module, 79, 127 range versus for loops, 388 range function, 342 range iterator, 367 support for multiple iterators, 369 rapid prototyping, 11 rational number objects, 129 raw string literals, 85 raw strings, 161 raw_input function (Python 2.x), 50 re (regular expression) module, 85 string handling in Python 3.0, 929 read method, 233 readline method, 98, 232 recursive functions, 465–469 arbitrary structures, handling, 468 coding alternatives, 466 direct or indirect, 467 loop statements, versus, 467 summation, 465 recursive imports, 603 reduce, 363 reduce function, 481 reference counters, 147 references, 145 copies of objects, versus, 241 shared references, 148–152 equality, 151 in-place changes, 149 relative imports, 561 reload function, 52, 554–558 example, 556 from imports and, 601 interactive testing, 602 import and from, contrasted with, 555 transitive module reloads, 595–598 usage notes, 56 version 3.0 requirements, 53 reloadall.py, 596 remove method, 87, 206 repetition, 81 replace method, 83 repr function, 79 str, compared to, 116 return statement, 397, 398, 404 reverse method, 87, 205 round function, 126 1156 | Index rstrip method, 235 S SAX parsing, 935 scientific programming, 11 ScientificPython programming extensions, 12 SciPy programming extensions, 12, 140 scopes, 407–414 basics, 407 built-in scope, 412 defaults with loop variables, versus, 423 example, 411 global statements, 408 module files and, 416 name resolution and the LEGB rule, 410 namespaces, 407 nested functions and, 419–425 nonlocal, 408 rules, 408 script0.py, 24 script1.py, 42 running with an import, 51 scripts, 5, 42 search tables (see dictionaries) self argument, 685 semicolon (;), 265 send method, 496 sequence assignment statements, 280 sequence assignments, 281–284 sequence operations, 80 sequences, 80 set comprehensions, 507 set numeric type, 133–139 dictionaries, compared to, 135 immutable constraints and frozen sets, 136 Python 2.6, 133 Python 3.0, 135 set comprehensions in Python 3.0, 137 set object type, 99 setsubclass.py, 776 shared references, 148–152 equality, 151 in-place changes, 149 Shedskin C++ translator, 31, 889 shelve module, 670–675, 744 advantages and disadvantages, 673 database client, 672 database files, 672 object storage in shelve databases, 671 shelves and dictionaries, 670 updating a shelve’s objects, 674 simple functions, 796 single quotes (') and strings, 158 site module, 537 slice assignment in lists, 202 slice objects, 168 slicing, 81, 165, 166 example, 168 extended slicing, 167 loops, usage in, 343 software components, 653 sort method, 87, 203, 388, 454 sorted function, 93, 363 sorts in Python 3.0, 204 source code, 26 source file character set encoding declarations, 912 spaces, 314 special characters, 897 split method, 235 square brackets [ ], 78, 108, 269 dictionaries and, 209 list comprehensions and, 359, 486, 504 lists and, 89, 199 square roots, 126 stack trace, 828, 840 Stackless Python, 33 standard library, 4, 533 library directories, 537 standard manual set, 386 standard output stream (stdout), 297 state information, 102, 644 state retention, 429–432, 727, 997–1000 state with classes, 430 statements, 75, 261–276 assignment statements (see assignment statements) compound statements, 264 differences from other C-like languages, 265 end-of-line, 265 expression statements, 295 in-place changes, 296 indentation syntax, 266 multiline statements, 317 Python 3.0 statement set, 262 syntax, 264 block rule special case, 270 colon (:), 264 indentation, 266 interactive loops, 271–276 semicolons, 269 statement separators, 269 static methods, 686, 795–801 alternatives to, 798 coding with decorator syntax, 804 counting instances, 800 Python 2.6 and 3.0, 796 using, 799 staticmethod function, 799 steps, 167 StopIteration exception, 352 str, 79, 157, 169 repr, compared to, 116 str object type data encoding in, 901 str string type, 899 Python 2.6 operation set, 911 Python 3.0 compared to 2.x, 899 Unicode and, 85 stream processors, 742 stream redirection, 44 strides, 167 string exceptions, 859 deprecation of, 858 string formatting, 179–183 advanced expressions, 181 dictionary-based formatting expressions, 182 string formatting type codes, 181 string formatting method calls, 183–193 format method, 184–193 % formatting expression, compared to, 187 justification for, 190 keys, attributes, and offsets, 184 string methods, 172–179 additional examples, 177 changing strings example, 174 parsing text example, 176 Python 2.x string module, 178 string method calls to format method, 185 string method calls, Python 3.0, 173 string object type, 80–86, 155–163 coding special characters, 85 pattern matching, 85 raw string literals, 85 Index | 1157 sequence operations and, 80 string literals, 157–163 common literals and operations, 155 escape sequences, 158–161 raw strings and escapes, 161 single and double quoted strings, 158 string backslash characters, 159 triple quotes, 162 version 3.0 changes, 896 versions 2.6 and 3.0 string types, 157 string operations, 163–172 basic operations, 164 changing strings, 171 indexing, 165, 166 slicing, 165, 166 extended slicing, 167 string conversions, 169 character code conversions, 170 strings, 897 16- and 32-bit Unicode values, coding of, 907 ASCII text, coding, 905 bytearray objects, using, 917–920 bytes objects (Python 3.0), 913–917 method calls, 913 sequence operations, 914 bytes string type making bytes objects, 915 character encoding schemes and, 897 encoding conversions, 909 escape sequence coding by type, 908 examples of usage (Python 3.0), 902–904 literals and basic properties, 902 mixing string types, 916 mutability or immutability of string types, 903 non-ASCII text, coding, 905 non-ASCII text, encoding and decoding, 906 pickle object serialization module (Python 3.0), 932 Python 3.0 string types, usage, 920 re pattern matching module (Python 3.0), 929 source file character set encoding declarations, 912 string methods, 82 string types, 899 1158 | Index struct binary data module (Python 3.0), 930 text and binary files, 920 BOM in Python 3.0, 926 file modes in Python 3.0, 921 type and content mismatches, 923 Unicode in Python 2.6, 928 type conversions, 903 Unicode files, using, 924 reading and writing in Python 3.0, 924 unicode strings, coding, 904 Unicode strings, Python 2.6 coding, 910 XML parsing tools, 934 strong typing, 78 struct module string handling, Python 3.0, 930 subclasses, 614, 629, 653–658 coding, 653–658 augmenting methods, 654–656 inheritance, customization, and extension, 657 OOP, as illustration of, 658 polymorphism, 656 sum function, 125 Super class, 689 superclasses, 614, 629 abstract superclasses, 690–693 syntax rules, 264–271, 314–319 indentation, 315 multiline statements, 317 open syntactic pairs rule, 318 sys.exc_info, 880–882 sys.exit(statuscode) call, 883 sys.getdefaultencoding function, 896 sys.modules table, 534 sys.path list, 536, 538, 590 system namespace partitioning and modules, 530 systems programming, T termination actions, 830 testdriver function, 880 tester, 426 testing of code, 645 testmixin.py, 759 testprint30.py, 458 text files, 98, 233, 901, 920 in Python 3.0, 921 text-mode files, 920 text.py, 912 threenames.py, 54 time module, 509 alternatives, 513 timeit module, 517 timer module, keyword-only arguments, 516 tkinter, 59 getting support on Linux, 1094 settings, 1094 top-level code, 387 top-level file, 51, 531 transitive module reloads, 595–598 triple quotes, 162 True and False, 414 True and False Boolean values, 139 true and false values, 246 true division, 110, 117 truth tests, 320 try except statement, 831 try statement, 96, 263, 826, 840 (see also exceptions) debugging with, 879 except statement and, 828 nested try statements, 873–877 Python 2.5 and later, 835 try except/else, 835–842 try statement clause forms, 837–839 try/else clause, 839 try/finally statement, 842–843 coding termination actions, 843 unified try/except/finally, 844–847 example, 846 nesting finally and except, 845 statement syntax, 845 try/finally statement, 827, 830 file and server connection closure, 879 tuple object type, 96 tuple-unpacking assignment statements, 280 tuples, 114, 225–229 common literals and operations, 226 conversions, methods, and immutability, 228 in for loops, 336 immutability and tuple contents, 229 lists, compared to, 229 sorting, 228 supported sequence operations, 227 syntax with parentheses and commas, 227 type class, 1061 type hierarchies, 248 type object type, 100, 250, 1058 typesubclass.py, 775 U unbound methods, 750, 796 Python 3.0 status as functions, 752 undefined name exception, 691 underscore (_), 584 Unicode, 897 strings, coding of, 904 text, handling in versions 2.6 and 3.0, 896 Unicode files, 924 reading and writing (Python 3.0), 924 decoding mismatches, 925 file input decoding, 925 file output encoding, 925 manual encoding, 924 unicode string type (Python 2.6), 911 unicode string type (Python 2.x), 899, 910 unicode strings, 157 union function, 456 unit tests with name attribute, 586 unittest, 887 Unix env lookup trick, 47 executable scripts, 46 Python command line, starting, 36 Unladen Swallow project, 33 update method, 211 user base of Python language, user-defined classes, 101 user-defined exceptions, 830 UTF-8 encoding, 898 utility modules, 108 V values method, 211, 370 van Rossum, Guido, 14 variables, 113, 144–145 declaration, 114 initialization, 546 local variables, 404 scope, 408 variable name rules, 292–295 Index | 1159 W websites, 676 while loop, 94 versus for loops, 388 while statement, 327 range function and, 342 Windows automatic file extensions, 45 executable files, displaying output, 49 icon clicks for program initiation, 47 IDLE user interface and, 58 program files, opening with icons, 47 Python command line, starting in, 36 Python files, running in, 44 Python standard manual set, 386 Windows Notepad, file encoding specification, 926 with statement, 129, 842 with/as extension, 263 with/as statement, 832, 851–855 context management protocol, 853 usage, 852 wrapper classes, 745 wrapper objects, 984 wrappers, catching exceptions with, 882 write method, 232 X XML, 934 Y yield expression, 263 yield operator, 108 yield statement, 397, 399 usage in generators, 493 Z zip, 363, 365 zip function, 345 dictionary construction using, 347 zip iterator, 368 ZODB object-oriented database system, 676 1160 | Index About the Author Mark Lutz is the world leader in Python training, the author of Python’s earliest and bestselling texts, and a pioneering figure in the Python community Mark is the author of the popular O’Reilly books Learning Python, Programming Python, and Python Pocket Reference, all available in third or fourth editions in 2009 He has been using and promoting Python since 1992, started writing Python books in 1995, and began teaching Python classes in 1997 As of early 2009, Mark has instructed 225 Python training sessions, taught some 3,500 students, and written Python books that have sold roughly a quarter of a million copies and been translated into more than a dozen languages In addition, he holds B.S and M.S degrees in computer science from the University of Wisconsin, and during the last 25 years he has worked as a professional developer on compilers, programming tools, scripting applications, and assorted client/server systems Mark can be reached on the Web at http://www.rmi.net/~lutz Colophon The animal on the cover of Learning Python, Fourth Edition, is a wood rat (Neotoma Muridae) The wood rat lives in a wide range of conditions (mostly rocky, scrub, and desert areas) over much of North and Central America, generally at some distance from humans Wood rats are good climbers, nesting in trees or bushes up to six meters off the ground; some species burrow underground or in rock crevices or inhabit other species’ abandoned holes These grayish-beige, medium-size rodents are the original pack rats: they carry anything and everything into their homes, whether or not it’s needed, and are especially attracted to shiny objects such as tin cans, glass, and silverware The cover image is a 19th-century engraving from Cuvier’s Animals The cover font is Adobe ITC Garamond The text font is Linotype Birka; the heading font is Adobe Myriad Condensed; and the code font is LucasFont’s TheSansMonoCondensed ... Test Your Knowledge: Answers 64 4 64 4 64 5 64 6 64 8 64 9 65 1 6 52 65 3 65 3 65 4 65 4 65 6 65 7 65 8 65 8 66 0 66 0 66 3 66 4 66 5 66 6 66 7 66 8 66 9 67 0 67 1 6 72 67 4 67 5 67 7 67 7 67 8 28 Class Coding Details ... Knowledge: Part II Exercises 22 5 22 7 22 9 22 9 23 0 23 1 23 2 23 8 23 9 24 1 24 1 24 4 24 6 24 6 24 8 24 9 25 0 25 1 25 1 25 2 25 2 25 3 25 3 25 4 25 4 25 5 Part III Statements and Syntax 10 Introducing Python Statements ... Knowledge: Answers 32 7 32 8 32 8 32 9 32 9 33 0 33 1 33 1 3 32 33 4 33 4 33 5 34 1 3 42 34 3 34 4 34 5 34 8 34 9 34 9 35 0 14 Iterations and Comprehensions, Part 35 1 Iterators: A