If you do not wish to import division from __future__ module in your code but want true division to always prevail, you can use the -Qnew switch. There are also other options for using -Q, as summarized in Table D.1.
For example, the -Qwarnall option is used in the Tools/scripts/
fixdiv.py script found in the Python source distribution.
As you may have guessed by now, all of the transition efforts have already been implemented in Python 2.2, and no new 2.6-specific functionality has been added with respect to Python 3 migration. Table D.2 summarizes the division operators and their functionality in the various Python releases.
You can read more about the change to the division operator in PEP 238 as well as in an article called “Keeping Up with Python: The 2.2 Release” I wrote for Linux Journal back in July 2002.
Table D.1 Division Operation -Q Command-Line Options
Option Description
old Always perform classic division new Always perform true division
warn Warn against int/int and long/long operations warnall Warn against all use of /
Table D.2 Python Release Default Division Operator Functionality
Operator 2.1- 2.2+ 3.xa
a. The “3.x” column also applies to Python 2.2+ with -Qnew or the __future__.division import.
/ Classic Classic True
// Not applicable Floor Floor
ptg 1064 Appendix D
Built-in Functions
print Statement or print() Function
It’s no secret that one of the most common causes of breakage between Python 2.x and 3.x applications is the change in the print statement, which becomes a built-in function in version 3.x. This change allows print() to be more flexible, upgradeable, and swappable if desired.
Python 2.6+ supports either the print statement or the print() built-in function. The default is the former usage, as it should be in a 2.x language. To discard the print statement and go with only the function in a “Python 3 mode” application, you would simply import print_function from __future__:
>>> print 'foo', 'bar' foo bar
>>>
>>> from __future__ import print_function
<built-in function print>
>>> print('foo', 'bar') foo bar
>>> print('foo', 'bar', sep='-') foo-bar
The preceding example demonstrates the power of print() being a func- tion. Using the print statement, we display the strings "foo" and "bar" to the user, but we cannot change the default delimiter or separator between strings, which is a space. In contrast, print()makes this functionality avail- able in its call as the argument sep, which replaces the default—and allows print to evolve and progress.
Note that this is a “one-way” import, meaning that there is no way to revert print() back to a function. Even issuing a "del print_function" will not have any effect. This major change is detailed in PEP 3105.
reduce() Moved to functools Module
In Python 3.x, the reduce()function, which is neither readily understood nor commonly used by many programmers today, has been “demoted” (much to the chagrin of many Python functional programmers) from being a built-in function to become a functools module attribute. It is available in func- tools beginning in 2.6.
ptg Migrating to Python 3 Starts with 2.6 1065
>>> from operator import add
>>> reduce(add, range(5)) 10
>>>
>>> import functools
>>> functools.reduce(add, range(5)) 10
Other Updates
One key theme in Python 3.x is the migration to greater use of iterators, espe- cially for built-in functions and methods that have historically returned lists.
Still other iterators are changing because of the updates to integers. The fol- lowing are the most high-profile built-in functions changed in Python 3.x:
• range()
• zip()
• map()
• filter()
• hex()
• oct()
Starting in Python 2.6, programmers can access the new and updated functions by importing the future_builtins module. Here is an example demonstrating both the old and new oct() and zip() functions:
>>> oct(87) '0127'
>>>
>>> zip(range(4), 'abcd')
[(0, 'a'), (1, 'b'), (2, 'c'), (3, 'd')]
>>> dict(zip(range(4), 'abcd')) {0: 'a', 1: 'b', 2: 'c', 3: 'd'}
>>>
>>> import future_builtins
>>> future_builtins.oct(87) '0o127'
>>>
>>> future_builtins.zip(range(4), 'abcd')
<itertools.izip object at 0x374080>
>>> dict(future_builtins.zip(range(4), 'abcd')) {0: 'a', 1: 'b', 2: 'c', 3: 'd'}
If you want to use only the Python 3.x versions of these functions in your current Python 2.x environment, you can override the old ones by importing
ptg 1066 Appendix D
all the new functions into your namespace. The following example demon- strates this process with oct():
>>> from future_builtins import *
>>> oct(87) '0o127'
Object-Oriented Programming:
Two Different Class Objects
Python’s original classes are now called “classic classes.” They had many flaws and were eventually replaced by “new-style” classes. The transition began in Python 2.2 and continues today.
Classic classes have the following syntax:
class ClassicClass:
pass
New-style classes have this syntax:
class NewStyleClass(object):
pass
New-style classes feature so many more advantages than classic classes that the latter have been preserved only for backward-compatibility purposes and are eliminated entirely in Python 3. With new-style classes, types and classes are finally unified (see Guido’s “Unifying Types and Classes in Python 2.2” essay as well as PEP 252 and PEP 253).
There are no new changes added in Python 2.6 for migration purposes.
Just be aware that all 2.2+ versions serve as hybrid interpreters, allowing for both class objects and instances of those classes. In Python 3, both syntaxes shown in the preceding examples result only in new-style classes being cre- ated. This behavior does not pose a serious porting issue, but you do need to be aware that classic classes don’t exist in Python 3.
Strings
One especially notable change in Python 3.x is that the default string type is changing. Python 2.x supports both ASCII and Unicode strings, with ASCII being the default. This support is swapped in Python 3: Unicode becomes the
ptg Migrating to Python 3 Starts with 2.6 1067
default, and ASCII strings are now called bytes. The bytes data structure contains byte values and really shouldn’t be considered a string (anymore) as much as it is an immutable byte array that contains data.
Current string literals will now require a leading b or B in Python 3.x, and current Unicode string literals will drop their leading u or U. The type and built-in function names will change from str to bytes and from unicode to str. In addition, there is a new mutable “string” type called bytearray that, likebytes, is also a byte array, only mutable.
You can find out more about using Unicode strings in the HOWTO and learn about the changes coming to string types in PEP 3137. Refer to Table C.1 for a chart on the various string types in both Python 2 and Python 3.
bytes Literals
To smooth the way for using bytes objects in Python 3.x, you can optionally prepend a regular ASCII/binary string in Python 2.6 with a leading b or B, thereby creating bytes literals (b'' or B'') as synonyms for str literals (''). The leading indicator has no bearing on any str object itself or any of the object’s operations (it is purely decorative), but it does prepare you for sit- uations in Python 3 where you need to create such a literal. You can find out more about bytes literals in PEP 3112
bytes is str
It should not require much of a stretch of the imagination to recognize that if bytes literals are supported, then bytes objects themselves need to exist in Python 2.6. Indeed, the bytes type is synonymous with str, so much so that
>>> bytes is str True
Thus you can use bytes or bytes()in Python 2.6 wherever you use str or str(). Further information on bytes objects can be found in PEP 358.
Exceptions
Python 2.6 has several features that allow for porting of exception handling and raising exceptions in Python 3.x.
ptg 1068 Appendix D
Handling Exceptions (Using as )
Python 3’s syntax for catching and handling a single exception looks like this:
except ValueError as e:
Thee variable contains the instance of the exception that provides the reason why the error was thrown. It is optional, as is the entire as e phrase. Thus this change really applies only to those users who save this value.
The equivalent Python 2 syntax uses a comma instead of the as keyword:
except ValueError, e:
This change was made in Python 3.x because of the confusion that occurs when programmers attempt to handle more than one exception with the same handler.
To catch multiple exceptions with the same handler, beginners often write this (invalid) code:
except ValueError, TypeError, e:
In fact, if you are trying to catch more than one exception, you need to use a tuple containing the exceptions:
except (ValueError, TypeError), e:
The as keyword in Python 3.0 (and 2.6+) is intended to ensure that the comma in the original syntax is no longer a source of confusion. However, the parentheses are still required when you are trying to catch more than one type of exception using the same handler:
except (ValueError, TypeError) as e:
For porting efforts, Python 2.6+ accepts either the comma or as when defining exception handlers that save the instance. In contrast; only the idiom withas is permitted in Python 3. More information about this change can be found in PEP 3110.
Raising Exceptions
The change in raising exceptions found in Python 3.x really isn’t a change at all; in fact, it doesn’t even have anything to do with the transition efforts asso- ciated with Python 2.6. Python 3’s syntax for raising exceptions (providing the optional reason for the exception) looks like this:
raise ValueError('Invalid value')
ptg Migrating to Python 3 Starts with 2.6 1069
Long-time Python users have probably been using the following idiom (although both approaches are supported in all 2.x releases):
raise ValueError, 'Invalid value'
To emphasize that raising exceptions is equivalent to instantiating an exception class and to provide some additional flexibility, Python 3 supports only the first idiom. The good news is that you don’t have to wait until you adopt 2.6 to start using this technique—the syntax with parentheses has actu- ally been valid since the Python 1 days.
Other Transition Tools and Tips
In addition to Python 2.6, developers have access to an array of tools that can make the transition to Python 3.x go more smoothly—in particular, the -3 switch (which provides obsolescence warnings) and the 2to3 tool (read more about it at http://docs.python.org/3.0/library/2to3.html). However, the most important tool that you can “write” is a good transition plan. In fact, there’s no substitute for planning.
Clearly, the Python 3.x changes do not represent some wild mutation of the familiar Python syntax. Instead, the variations are just enough to break the old code base. Of course, the changes will affect users, so a good transi- tion plan is essential. Most good plans come with tools or aids to help you out in this regard. The porting recommendations in the “What’s New in Python 3.0” document specifically state that good testing of code is critical, in addi- tion to the use of key tools (i.e., the 2to3 code conversion tool and Python 2.6). Without mincing words, here is exactly what is suggested at http://
docs.python.org/3.0/whatsnew/3.0.html#porting-to-python-3-0:
1. (Prerequisite) Start with excellent test coverage.
2. Port to Python 2.6. This should involve no more work than the average port from Python 2.x to Python 2.(x+1). Make sure that all your tests pass.
3. (Still using 2.6) Turn on the -3 command-line switch. It enables warnings about features that will be removed (or will change) in Python 3.0. Run your test suite again, and fix any code that generates warnings. Make sure that all your tests still pass.
4. Run the 2to3 source-to-source translator over your source code tree. Run the result of the translation under Python 3.0.
Manually fix any remaining issues, and continue fixing problems until all tests pass again.
ptg 1070 Appendix D
Conclusion
We know big changes are coming in the next generation of Python, simply because 3.x code is backward incompatible with any older releases. The changes, although significant, won’t require entirely new ways of thinking for programmers—though there is obvious code breakage. To ease the transition period, current and future releases of the remainder of the 2.x interpreters will contain 3.x-backported features.
Python 2.6 is the first of the “dual-mode” interpreters that will allow users to start programming against the 3.0 code base. You can read more about all of the new 2.6 features (not just the ones applicable to the 3.x transition) in the “What’s New in Python 2.6” document. Python 2.6 runs all 2.x software as well as understands some 3.x code. In this way, the 2.6 release helps sim- plify the porting and migration process and eases users gently into the next generation of Python programming.
Online References
Wesley J. Chun, “Keeping Up with Python: The 2.2 Release,” July 2002, http://www.linuxjournal.com/article/5597
A. M. Kuchling (amk at amk.ca), “What’s New in Python 2.6,” December 2008, http://docs.python.org/whatsnew/2.6.html
PEP Index, http://www.python.org/dev/peps
Unicode HOWTO, http://docs.python.org/3.0/howto/unicode.html
Guido van Rossum (guido at python.org), “Unifying Types and Classes in Python 2.2” (second draft, published for 2.2.3), April 2002, http://
www.python.org/2.2.3/descrintro.html
Guido van Rossum (guido at python.org). “What’s New in Python 3.0,”
December 2008, http://docs.python.org/3.0/whatsnew/3.0.html
ptg
1071
@ (“at-sign”), 422
& (ampersand), 277
&= (ampersand equal sign), 279
* (asterisk), 35, 680–681
\ (backslash), 62
^ (caret), 278, 678, 680
$ (dollar sign), 678, 679 . (dot), 678, 689–690
** (double asterisk), 35, 133, 140
`` (double single quotes), 104 - (hyphen), 680
- (minus sign), 278 . (period), 678, 689–690
|= (pipe equals), 277
| (pipe symbol), 277, 678, 689
+ (plus sign), 35, 159, 176–177, 213–214, 233, 681
# (pound sign), 34, 62 / (slash), 35, 130–131, 569 { } (braces), 681
[ ] (brackets), 259, 679–680, 690
> , >= (greater than), 277
< , <= (less than), 277
<>,!= (“not equals” comparison operators), 36 2to3 tool, 1055, 1069–1070
A
\A (special character), 678 abs( ) built-in function, 143
absolute import statement, 494–495 abstract methods, 519
abstraction, 382, 516 access models, 115–116 access modes, 326, 327
ActiveX, 989. See also COM (Component Object Model)
adapters (database) about, 932–933 definition of, 923 examples of, 934–945
ptg 1072 Index
adapters (continued) Gadfly, 939–945
MySQL, 934–936, 939–945 ORMs, 946
PostgreSQL, 936–937 related modules, 958–959 SQLite, 937–945
add( ) method, 281, 282, 284 addition ( + ) operator, 35
AF_INET (socket family name), 716, 718, 730 AF_UNIX (socket family name), 715, 718, 730 aggregation, 517
all( ) built-in function, 310 alternation ( | ), 678
American Standard Code for Information Interchange.See ASCII
ancestors, 545, 546 and keyword, 36, 100, 101 anonymous functions, 439–441 any( ) built-in function, 310
AOP (aspect-oriented programming), 424 API (Application Programming Interface),
489, 922–923, 985–989 apilevel, 924–925
Application Programming Interface. See API apply( ) built-in function, 414, 441, 442 arguments
class methods, 543 command-line, 338–339 for exceptions, 372–375 arguments (default)
functions, 49, 413, 429–442 GUI development, 824 Tkinter widgets, 824
using with instantiation, 532–533 arguments (function)
decorators with/without, 423–424 default, 49, 413, 429–442
dictionary, keyword variable, 434–436 formal, 428–432
grouped, 413–414
keyword, 412–413, 434–436 optional, 48
positional, 428–429
variable argument objects, 436–439 variable-length, 433–439
arguments (method), 532–533 arithmetic game (example), 415–417
arithmetic operators. See mathematical operators arrays.See lists; tuples
ASCII, 144–145, 197–198, 497, 698 aspect-oriented programming (AOP), 424 assert statement, 389–390
AssertionError exceptions, 390, 391 assigning/assignment
augmented, 37, 65–66 dictionaries, 255 lists, 209 multiple, 66
“multuple,” 66–67 numbers, 121 operators, 64–67 set types, 273, 274 strings, 168–169 tuples, 231 variables, 37, 64–67 association (classes), 517 asterisk operator ( * ), 680–681 async module, 741
atomic storage, 112
“at-sign” ( @ ), 422
attr( ) built-in functions, 560–561 attribute(s)
built-in functions, 629 built-in methods, 632 built-in types, 536–537 class, 520–525
complex number, built-in, 126–127 definition of, 46
double underscore ( __ ), 586 file, 336–337
functions, 420–421, 629, 630 importing, 496
interfaces to data, 516 __metaclass__, 610–611 methods, 632, 633 module, 333
multi-platform development, 333 naming, 478, 513
object, 90 privacy, 585–586 Queue module, 811
simple lookup example, 554–555 socket module, 730–731 special class, 523–525
special methods for customizing classes, 566
ptg Index 1073
user-defined function, 630 user-defined methods, 633 using files to store, 604
using local to substitute for module, 81
AttributeError exception, 363–364, 391, 590, 600
augmented assignment, 37, 65–66, 569, 574–575 authentication handler for HTTP, 867–869 auto-loaded modules, 496
B
\b, \B (special characters), 679 backbone (network), 858 backslash ( \ ), 62
bank tellers server example, 713–714 base class, 50. See also parent/parent class base representation, 143–144
BaseException, 371, 372, 391, 394 BaseHTTPServer module, 907–909, 911 bases argument, 504
__bases__ attribute, 524, 525, 548 BASIC language, 10
Beazley, David, 981
BIFs.See functions (built-in) BIMs.See methods (built-in) binary function, 447–448 binary literals, 1054, 1061–1062 binding
methods, 540–541 namespaces, 480
bit operators (integer-only), 135–136 Boa Constructor, 849
boilerplate, 968–974
bool( ) factory function, 137, 138
Boolean operators (and, or, not), 36, 100–101, 292 Boolean types, 93, 123, 145–147
bound methods, 522, 541 brace operators ( { } ), 681
bracket symbols ( [ ] ), 679–680, 690 break statement, 304–305, 308 BSD Unix, 12, 14, 16–17, 715 buffering (file), 327, 328 building Python, 13 __builtin__, 480–481, 496
built-in attributes, 126–127, 336–337 built-in exceptions, 391–393, 1040–1042 built-in functions. See functions (built-in) built-in methods (BIMs). See methods (built-in)
“built-in” names, 69 built-in types, 91, 536–537 __builtins__, 69, 480–481, 496 Button widget (Tk), 825–831 byte type, 116
bytes type, 1051–1052, 1067–1068 bytearray type, 1051–1052, 1067–1068
C
C language
conversion to/from, 969, 970
“danglingelse” statements, 293 extensions written in, 8
fopen( ), 326–327 Python and, 6, 8–9, 23, 26 varargs, 433
C# language, 27, 969. See also IronPython C++ language, 8, 23, 25, 26, 969, 970 __call__( ) special method, 634–635
“call by reference,” 48
callable( ) built-in function, 636, 637, 831 callable class, 803–805
callbacks, 823, 824
calling functions, 49, 412–417 arguments, 412–414, 436–439 built-in, 427
default arguments, 413 example, 415–417 function operator, 412 grouped arguments, 413–414 keyword arguments, 412–413 logging with closures, 461–463
with variable argument objects, 436–439 calling modules, 52–53
Canvas widget (Tk), 825 caret symbol ( ^ ), 678
case statement, proxy for, 294–295 case-insensitive import, 496–497 case-sensitive identifiers, 68 casting of sequences, 165–166 CGI (Common Gateway Interface)
about, 875–877
advanced example of, 896–906 applications, 877–892 building applications, 878–892 cgi module, 878
cookies, 895–897, 900–901, 906 creating static form Web page, 879–881
ptg 1074 Index
CGI (continued) file uploading, 894
generating form page, 882–886 generating results page, 881–886 HTTP headers, 882
multipart form submission, 894 multivalued fields, 895
setting up a Web server, 878–879 Unicode, 892–893
user inputs/error processing, 886–892 Web servers, 878–879, 906–909 cgi module, 878
CGIHTTPServer module, 907, 908, 911 char type (unsupported), 116, 207 character(s)
accessing, in strings, 169
matching any single ( . ), 678, 689–690 removing, 169–170
character classes ( [ ] ), 679–680, 690 characters, special, 208
and ASCII characters, 698
escaping using triple quotes, 38, 192–193 regular expressions, 676–677, 682, 690–693 repetition/grouping and, 690–693
representing character sets, 682 strings, 192–193
Checkbutton widget (Tk), 825
child/child class, 545–548, 554, 555, 586, 823 chr( ) built-in function, 144, 145, 187 class attributes, 520–525
accessing, 537–539 __bases__, 524, 525, 548 __class__, 524, 525 data, 520–521 determining, 522–523 __dict__, 522–525 __doc__, 524, 525
instance attributes vs., 537–540 methods, 521–522
modifying, 540
__module__ attribute, 524, 525 persistence, 539–540
privacy of, 586 __slots__, 597 special, 523–525 class definition, 510–511 class keyword, 50, 504 class variables, 521
classes, 50–52
about, 504–507, 518–520 built-in functions, 558–564 callable objects, 634 composition, 544–545 creating, 51–52, 510–511, 519
customizing with special methods, 564–585 declaring, 50–51, 72, 518–520
definition vs. declaration of, 519–520 methods, 507–512, 521–522 mix-in, 556
as namespace containers, 506 naming, 513
related modules, 615–617 Web server, 909
wrapping, 587
classic classes, 503, 504, 542–543, 554–558.
See also new-style classes
classic division operator ( / ), 35, 130–131, 569 classmethod( ) built-in function, 543 clause, 63
clear( ) method, 281, 282, 284 client data, 875–878
clients
FTP, 752–754
GUI applications as, 821 Internet, 747–748 NNTP, 760–765 POP3, 775–777 SMTP, 775–777
TCP, 723–726, 736, 738–740 twisted reactor TCP, 738–740 UDP, 728–729
Web, 855–857, 859–875 Windows, 713
client/server architecture, 711–715, 821, 855–856
client-side programming, 989–991 close( ) built-in method, 332, 335 closures, 422, 456–463, 680–681
cmp( ) built-in function, 102, 103, 136, 137, 184–185, 215–216, 260–262
code
commenting, 34–35 indenting, 41 integration of, 963
interpreted/byte-compiled, 11 profiling of, 8, 84–85, 965
ptg Index 1075
running example, 22
runtime generation/execution of, 642–649 skipping, 365
wrapping, in boilerplate, 968–974 code objects, 94, 635–636
code reuse, 7, 964 codecs, 199, 205
coerce( ) built-in function, 139, 143 collisions (key), 268–269
colocated servers, 857, 858 columns (database), 920
COM (Component Object Model), 989–991 command line
arguments, 338–339, 490 FTP client program, 755 options, 15
running Python from, 14–17 switches, 653
commands (database), 920 comments, 34–35, 62, 69–70
Common Gateway Interface. See CGI compile( ) function, 94, 636, 637–638,
684–686
compiling, 11, 685–686, 964, 965, 974–975 complex( ) factory function, 137, 138 complex numbers, 37, 38, 126–127 complex statements, 62–63
Component Object Model. See COM composite objects, 112
composition, 516–517, 544–545 compound objects, 112 compound statements, 62–63
concatenation ( + ) sequence operator, 159, 176–177, 213–214, 233
conditional code execution, 647–649
conditional statements, 291–296, See also while statement (loops). See also loops
auxiliary statements for, 308 conditional expressions, 295–296
elif (aka else-if) statement, 294–295, 308 else statement, 292–294, 307–308, 378 if statement, 41–42
if statement, 291–292 multiple, 292
andpass statement, 306–307 single statements suites, 292 switch/case statement proxy, 294–295 connect( ) function, 925–927
connection objects, 927–928 connectionless sockets, 717–718 connection-oriented sockets, 716–717
constructors, 510, 511, 527–529, 532–533, 930–931 container storage, 112
context expression (context_expr), 384 context management (for exceptions), 382–385 continuation (exception handling), 361 continuation ( \ ) statements, 62 continue statement, 305–306, 308 conversion
ASCII functions for, 144–145 codes, 969, 970
factory functions, 137–138 sequences, 165–166 symbols, 178–181, 1029
cookies, 856, 895–897, 900–901, 906
copy( ) function/method, 240–241, 267, 283 counting loops, 297
couroutines, 467–468 cPickle, 349, 350 cProfile module, 85 CPython, 26, 27
credit card transaction data example, 375–378, 380–381
cross-product generator expressions example, 317 currying, 450
cursor (database), 920 cursor objects, 929–930 customizing and extensions, 964
CWI (Centrum voor Wiskunde en Informatica), 6 cycles (import), 497–500
cyclic reference, 79
D
daemon threads, 801
“danglingelse”, avoiding, 63, 293–294 data attributes. See attribute(s)
data descriptors, 599 data hiding, 516 data structures, 223–230
database programming, 919–959. See also DB-API (Python Database Application Programmer’s Interface)
about, 920–922
adapter examples, 934–945 components of databases, 920 Gadfly, 923, 939–945