Fluent Python CLEAR, CONCISE, AND EFFECTIVE PROGRAMMING Luciano Ramalho WOW! eBook www.wowebook.org Fluent Python Python’s simplicity lets you become productive quickly, but this often means you aren’t using everything it has to offer With this hands-on guide, you’ll learn how to write effective, idiomatic Python code by leveraging its best—and possibly most neglected—features Author Luciano Ramalho takes you through Python’s core language features and libraries, and shows you how to make your code shorter, faster, and more readable at the same time Many experienced programmers try to bend Python to fit patterns they learned from other languages, and never discover Python features outside of their experience With this book, those Python programmers will thoroughly learn how to become proficient in Python proud to have been “ Iaam tech reviewer for this excellent book—not only will it help many intermediate Python programmers on their road towards mastery, but it has taught me quite a few things, too! ” This book covers: —Alex Martelli ■■ The Python data model: understand how special methods are the key to the consistent behavior of objects ■■ Data structures: take full advantage of built-in types, and understand the text versus bytes duality in the Unicode age ■■ Functions as objects: view Python functions as first-class objects, and understand how this affects popular design patterns ■■ Object-oriented idioms: build classes by learning about references, mutability, interfaces, operator overloading, and multiple inheritance ■■ Control flow: leverage context managers, generators, coroutines, and concurrency with the concurrent.futures and asyncio packages ■■ Python Software Foundation Fellow is a “treasure Fluent Python trove full of useful programming tricks for intermediate to advanced Python coders who want to push the boundaries of their knowledge ” —Daniel and Audrey Roy Greenfeld authors of Two Scoops of Django Metaprogramming: understand how properties, attribute descriptors, class decorators, and metaclasses work Luciano Ramalho, a Python programmer since 1998, is a Python Software Foundation fellow, co-owner of Python.pro.br—a training company in Brazil— and cofounder of Garoa Hacker Clube, Brazil’s first hackerspace He has led software development teams and taught Python courses in Brazilian media, banking, and government sectors PROGR AMMING/PY THON US $49.99 Twitter: @oreillymedia facebook.com/oreilly CAN $57.99 ISBN: 978-1-491-9-46008 WOW! eBook www.wowebook.org Fluent Python Luciano Ramalho Boston WOW! eBook www.wowebook.org Fluent Python by Luciano Ramalho Copyright © 2015 Luciano Gama de Sousa Ramalho All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://safaribooksonline.com) For more information, contact our corporate/ institutional sales department: 800-998-9938 or corporate@oreilly.com Editors: Meghan Blanchette and Rachel Roumeliotis Production Editor: Melanie Yarbrough Copyeditor: Kim Cofer Proofreader: Jasmine Kwityn August 2015: Indexer: Judy McConville Cover Designer: Ellie Volckhausen Interior Designer: David Futato Illustrator: Rebecca Demarest First Edition Revision History for the First Edition: 2015-07-24: First release 2015-08-21: Second release See http://oreilly.com/catalog/errata.csp?isbn=9781491946008 for release details The O’Reilly logo is a registered trademark of O’Reilly Media, Inc Fluent Python, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc While the publisher and author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work Use of the information and instructions contained in this work is at your own risk If any code samples or other technology this work contains or describes is subject to open source licenses or the intel‐ lectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights ISBN: 978-1-491-94600-8 [LSI] WOW! eBook www.wowebook.org Para Marta, com todo o meu amor WOW! eBook www.wowebook.org WOW! eBook www.wowebook.org Table of Contents Preface xv Part I Prologue The Python Data Model A Pythonic Card Deck How Special Methods Are Used Emulating Numeric Types String Representation Arithmetic Operators Boolean Value of a Custom Type Overview of Special Methods Why len Is Not a Method Chapter Summary Further Reading Part II 11 12 12 13 14 14 15 Data Structures An Array of Sequences 19 Overview of Built-In Sequences List Comprehensions and Generator Expressions List Comprehensions and Readability Listcomps Versus map and filter Cartesian Products Generator Expressions Tuples Are Not Just Immutable Lists Tuples as Records Tuple Unpacking 20 21 21 23 23 25 26 26 27 v WOW! eBook www.wowebook.org Nested Tuple Unpacking Named Tuples Tuples as Immutable Lists Slicing Why Slices and Range Exclude the Last Item Slice Objects Multidimensional Slicing and Ellipsis Assigning to Slices Using + and * with Sequences Building Lists of Lists Augmented Assignment with Sequences A += Assignment Puzzler list.sort and the sorted Built-In Function Managing Ordered Sequences with bisect Searching with bisect Inserting with bisect.insort When a List Is Not the Answer Arrays Memory Views NumPy and SciPy Deques and Other Queues Chapter Summary Further Reading 29 30 32 33 33 34 35 36 36 37 38 40 42 44 44 47 48 48 51 52 55 57 59 Dictionaries and Sets 63 Generic Mapping Types dict Comprehensions Overview of Common Mapping Methods Handling Missing Keys with setdefault Mappings with Flexible Key Lookup defaultdict: Another Take on Missing Keys The missing Method Variations of dict Subclassing UserDict Immutable Mappings Set Theory set Literals Set Comprehensions Set Operations dict and set Under the Hood A Performance Experiment Hash Tables in Dictionaries vi | Table of Contents WOW! eBook www.wowebook.org 64 66 66 68 70 70 72 75 76 77 79 80 81 82 85 85 87 Practical Consequences of How dict Works How Sets Work—Practical Consequences Chapter Summary Further Reading 90 93 93 94 Text versus Bytes 97 Character Issues Byte Essentials Structs and Memory Views Basic Encoders/Decoders Understanding Encode/Decode Problems Coping with UnicodeEncodeError Coping with UnicodeDecodeError SyntaxError When Loading Modules with Unexpected Encoding How to Discover the Encoding of a Byte Sequence BOM: A Useful Gremlin Handling Text Files Encoding Defaults: A Madhouse Normalizing Unicode for Saner Comparisons Case Folding Utility Functions for Normalized Text Matching Extreme “Normalization”: Taking Out Diacritics Sorting Unicode Text Sorting with the Unicode Collation Algorithm The Unicode Database Dual-Mode str and bytes APIs str Versus bytes in Regular Expressions str Versus bytes on os Functions Chapter Summary Further Reading Part III 98 99 102 103 105 105 106 108 109 110 111 114 117 119 120 121 124 126 127 129 129 130 132 133 Functions as Objects First-Class Functions 139 Treating a Function Like an Object Higher-Order Functions Modern Replacements for map, filter, and reduce Anonymous Functions The Seven Flavors of Callable Objects User-Defined Callable Types Function Introspection 140 141 142 143 144 145 146 Table of Contents WOW! eBook www.wowebook.org | vii From Positional to Keyword-Only Parameters Retrieving Information About Parameters Function Annotations Packages for Functional Programming The operator Module Freezing Arguments with functools.partial Chapter Summary Further Reading 148 150 154 156 156 159 161 162 Design Patterns with First-Class Functions 167 Case Study: Refactoring Strategy Classic Strategy Function-Oriented Strategy Choosing the Best Strategy: Simple Approach Finding Strategies in a Module Command Chapter Summary Further Reading 168 168 172 175 176 177 179 180 Function Decorators and Closures 183 Decorators 101 When Python Executes Decorators Decorator-Enhanced Strategy Pattern Variable Scope Rules Closures The nonlocal Declaration Implementing a Simple Decorator How It Works Decorators in the Standard Library Memoization with functools.lru_cache Generic Functions with Single Dispatch Stacked Decorators Parameterized Decorators A Parameterized Registration Decorator The Parameterized Clock Decorator Chapter Summary Further Reading viii | Table of Contents WOW! eBook www.wowebook.org 184 185 187 189 192 195 196 198 199 200 202 205 206 206 209 211 212 ... professional Python programmers are using Python 2, so I took special care to highlight Python features that may be new to that audience However, Fluent Python is about making the most of Python 3.4,... $57.99 ISBN: 978-1-491-9-46008 WOW! eBook www.wowebook.org Fluent Python Luciano Ramalho Boston WOW! eBook www.wowebook.org Fluent Python by Luciano Ramalho Copyright © 2015 Luciano Gama de Sousa... For This book was written for practicing Python programmers who want to become pro‐ ficient in Python If you know Python but are willing to migrate to Python 3.4 or later, you should be fine At