Tài liệu Python in a nutshell, 2nd edition

www.it-ebooks.info www.it-ebooks.info PYTHON IN A NUTSHELL Second Edition Alex Martelli Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo www.it-ebooks.info Python in a Nutshell, Second Edition by Alex Martelli Copyright © 2006, 2003 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (safari.oreilly.com). For more information, contact our corporate/institutional sales department: (800) 998-9938 or [email protected]. Editor: Mary T. O’Brien Production Editor: Matt Hutchinson Copyeditor: Linley Dolby Proofreader: Matt Hutchinson Indexer: Johnna Dinse Cover Designer: Emma Colby Interior Designer: Brett Kerr Cover Illustrator: Karen Montgomery Illustrators: Robert Romano and Jessamyn Read Printing History: March 2003: First Edition. July 2006: Second Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. The In a Nutshell series designations, Python in a Nutshell, the image of an African rock python, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. ISBN: 978-0596-10046-9 [LSI] [2011-07-01] www.it-ebooks.info Chapter 1 Table of Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Part I. Getting Started with Python 1. Introduction to Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 The Python Language The Python Standard Library and Extension Modules Python Implementations Python Development and Versions Python Resources 3 5 5 8 9 2. Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Installing Python from Source Code Installing Python from Binaries Installing Jython Installing IronPython 14 18 20 21 3. The Python Interpreter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 The python Program Python Development Environments Running Python Programs The jython Interpreter The IronPython Interpreter 22 26 28 29 30 iii www.it-ebooks.info Part II. Core Python Language and Built-ins 4. The Python Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Lexical Structure Data Types Variables and Other References Expressions and Operators Numeric Operations Sequence Operations Set Operations Dictionary Operations The print Statement Control Flow Statements Functions 33 38 46 50 52 53 58 59 61 62 70 5. Object-Oriented Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Classes and Instances Special Methods Decorators Metaclasses 82 104 115 116 6. Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 The try Statement Exception Propagation The raise Statement Exception Objects Custom Exception Classes Error-Checking Strategies 121 126 128 129 132 134 7. Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Module Objects Module Loading Packages The Distribution Utilities (distutils) 139 144 149 150 8. Core Built-ins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Built-in Types Built-in Functions The sys Module The copy Module The collections Module iv | Table of Contents www.it-ebooks.info 154 158 168 172 173 The functional Module The bisect Module The heapq Module The UserDict Module The optparse Module The itertools Module 175 176 177 178 179 183 9. Strings and Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 Methods of String Objects The string Module String Formatting The pprint Module The repr Module Unicode Regular Expressions and the re Module 186 191 193 197 198 198 201 Part III. Python Library and Extension Modules 10. File and Text Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Other Chapters That Also Deal with Files Organization of This Chapter File Objects Auxiliary Modules for File I/O The StringIO and cStringIO Modules Compressed Files The os Module Filesystem Operations Text Input and Output Richer-Text I/O Interactive Command Sessions Internationalization 215 215 216 224 229 230 240 241 256 258 265 269 11. Persistence and Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Serialization DBM Modules Berkeley DB Interfacing The Python Database API (DBAPI) 2.0 278 285 288 292 Table of Contents www.it-ebooks.info | v 12. Time Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 The time Module The datetime Module The pytz Module The dateutil Module The sched Module The calendar Module The mx.DateTime Module 302 306 313 313 316 317 319 13. Controlling Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328 Dynamic Execution and the exec Statement Internal Types Garbage Collection Termination Functions Site and User Customization 328 331 332 337 338 14. Threads and Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 Threads in Python The thread Module The Queue Module The threading Module Threaded Program Architecture Process Environment Running Other Programs The mmap Module 341 341 342 344 350 353 354 360 15. Numeric Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365 The math and cmath Modules The operator Module Random and Pseudorandom Numbers The decimal Module The gmpy Module 365 368 370 372 373 16. Array Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 The array Module Extensions for Numeric Array Computation The Numeric Package Array Objects Universal Functions (ufuncs) Auxiliary Numeric Modules vi | Table of Contents www.it-ebooks.info 375 377 378 378 399 403 17. Tkinter GUIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Tkinter Fundamentals Widget Fundamentals Commonly Used Simple Widgets Container Widgets Menus The Text Widget The Canvas Widget Layout Management Tkinter Events 406 408 415 420 423 426 436 442 446 18. Testing, Debugging, and Optimizing . . . . . . . . . . . . . . . . . . . . . . . . . 451 Testing Debugging The warnings Module Optimization 452 461 471 474 Part IV. Network and Web Programming 19. Client-Side Network Protocol Modules . . . . . . . . . . . . . . . . . . . . . . . 493 URL Access Email Protocols The HTTP and FTP Protocols Network News Telnet Distributed Computing Other Protocols 493 503 506 511 515 517 519 20. Sockets and Server-Side Network Protocol Modules . . . . . . . . . . . . 520 The socket Module The SocketServer Module Event-Driven Socket Programs 521 528 533 21. CGI Scripting and Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 CGI in Python Cookies Other Server-Side Approaches 546 553 557 Table of Contents www.it-ebooks.info | vii 22. MIME and Network Encodings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561 Encoding Binary Data as Text MIME and Email Format Handling 561 564 23. Structured Text: HTML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575 The sgmllib Module The htmllib Module The HTMLParser Module The BeautifulSoup Extension Generating HTML 576 580 583 585 586 24. Structured Text: XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591 An Overview of XML Parsing Parsing XML with SAX Parsing XML with DOM Changing and Generating XML 592 593 598 606 Part V. Extending and Embedding 25. Extending and Embedding Classic Python . . . . . . . . . . . . . . . . . . . . 613 Extending Python with Python’s C API Extending Python Without Python’s C API Embedding Python Pyrex 614 645 647 650 26. Extending and Embedding Jython . . . . . . . . . . . . . . . . . . . . . . . . . . . 655 Importing Java Packages in Jython Embedding Jython in Java Compiling Python into Java 656 659 662 27. Distributing Extensions and Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 666 Python’s distutils py2exe py2app cx_Freeze PyInstaller 666 675 676 676 676 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 677 viii | Table of Contents www.it-ebooks.info Chapter 2 Preface The Python programming language manages to reconcile many apparent contradictions: it’s both elegant and pragmatic, it’s both simple and powerful, it’s very high-level yet doesn’t get in your way when you need to fiddle with bits and bytes, it’s suitable for programming novices and great for experts, too. This book is aimed at programmers with some previous exposure to Python, as well as experienced programmers coming to Python for the first time from other programming languages. The book is a quick reference to Python itself, the most commonly used parts of its vast standard library, and some of the most popular and useful third-party modules and packages, covering a wide range of application areas, including web and network programming, GUIs, XML handling, database interactions, and high-speed numeric computing. The book focuses on Python’s cross-platform capabilities and covers the basics of extending Python and embedding it in other applications, using either C or Java™. How This Book Is Organized This book has five parts, as follows. Part I, Getting Started with Python Chapter 1, Introduction to Python Covers the general characteristics of the Python language and its implementations, and discusses where to get help and information. Chapter 2, Installation Explains how to obtain and install Python on your computer(s). Chapter 3, The Python Interpreter Covers the Python interpreter program, its command-line options, and how it is used to run Python programs and in interactive sessions. The chapter also ix www.it-ebooks.info mentions text editors that are particularly suitable for editing Python programs and auxiliary programs for thoroughly checking your Python sources, and examines some full-fledged integrated development environments, including IDLE, which comes free with standard Python. Part II, Core Python Language and Built-ins Chapter 4, The Python Language Covers Python syntax, built-in data types, expressions, statements, and how to write and call functions. Chapter 5, Object-Oriented Python Explains object-oriented programming in Python. Chapter 6, Exceptions Covers how to deal with errors and abnormal conditions in Python programs. Chapter 7, Modules Covers how Python lets you group code into modules and packages, how to define and import modules, and how to install third-party Python extensions that are packaged in standard Python ways. Chapter 8, Core Built-ins Refers to built-in data types and functions, and some of the most fundamental modules in the standard Python library (roughly, modules supplying functionality that, in some other languages, is built into the language itself). Chapter 9, Strings and Regular Expressions Covers Python’s powerful string-processing facilities, including Unicode strings and regular expressions. Part III, Python Library and Extension Modules Chapter 10, File and Text Operations Explains how to deal with files and text processing using built-in Python file objects, many modules from Python’s standard library, and platform-specific extensions for rich text I/O. The chapter also covers issues of internationalization and localization, and the specific task of defining interactive textmode command sessions with Python. Chapter 11, Persistence and Databases Introduces Python’s serialization and persistence mechanisms, as well as Python’s interfaces to DBM databases, the Berkeley Database, and relational (SQL-based) databases. Chapter 12, Time Operations Covers how to deal with times and dates in Python, using the standard library and popular extensions. Chapter 13, Controlling Execution Explains how to achieve advanced execution control in Python, including execution of dynamically generated code and control of garbage-collection operations. The chapter also covers some Python internal types, and the x | Preface www.it-ebooks.info specific issue of registering “clean-up” functions to be executed at programtermination time. Chapter 14, Threads and Processes Covers Python’s functionality for concurrent execution, both via multiple threads running within one process and via multiple processes running on a single machine. The chapter also covers how to access the process’s environment, and how to access files via memory-mapping mechanisms. Chapter 15, Numeric Processing Shows Python’s features for numeric computations, both in standard library modules and in third-party extension packages; in particular, the chapter covers how to use decimal floating-point numbers instead of the default binary floating-point numbers. The chapter also covers how to get and use pseudorandom and truly random numbers. Chapter 16, Array Processing Covers built-in and extension packages for array handling, focusing on the traditional Numeric third-party extension, and mentions other, more recently developed alternatives. Chapter 17, Tkinter GUIs Explains how to develop graphical user interfaces in Python with the Tkinter package included with the standard Python distribution, and briefly mentions other alternative Python GUI frameworks. Chapter 18, Testing, Debugging, and Optimizing Deals with Python tools and approaches that help ensure your programs are correct (i.e., that your programs do what they’re meant to do), find and correct errors in your programs, and check and enhance your programs’ performance. The chapter also covers the concept of “warning” and the Python library module that deals with it. Part IV, Network and Web Programming Chapter 19, Client-Side Network Protocol Modules Covers many modules in Python’s standard library that help you write network client programs, particularly by dealing with various network protocols from the client side and handling URLs. Chapter 20, Sockets and Server-Side Network Protocol Modules Explains Python’s interfaces to low-level network mechanisms (sockets), standard Python library modules that help you write network server programs, and asynchronous (event-driven) network programming with standard modules and the powerful Twisted extension. Chapter 21, CGI Scripting and Alternatives Covers the basics of CGI programming, how to perform CGI programming in Python with standard Python library modules, and how to use “cookies” to deal with session-state in HTTP server-side programming. The chapter also mentions many alternatives to CGI programming for server-side web programming through popular Python extensions. Preface | www.it-ebooks.info xi Chapter 22, MIME and Network Encodings Shows how to process email and other network-structured and encoded documents in Python. Chapter 23, Structured Text: HTML Covers Python library modules that let you process and generate HTML documents. Chapter 24, Structured Text: XML Covers Python library modules and popular extensions that let you process, modify, and generate XML documents. Part V, Extending and Embedding Chapter 25, Extending and Embedding Classic Python Shows how to code Python extension modules using C and other classic compiled languages, how to embed Python in applications coded in such languages, and alternative ways to extend Python and access existing C, C++, and Fortran libraries. Chapter 26, Extending and Embedding Jython Shows how to use Java classes from the Jython implementation of Python, and how to embed Jython in applications coded in Java. Chapter 27, Distributing Extensions and Programs Covers the tools that let you package Python extensions, modules, and applications for distribution. Conventions Used in This Book The following conventions are used throughout this book. Reference Conventions In the function/method reference entries, when feasible, each optional parameter is shown with a default value using the Python syntax name=value. Built-in functions need not accept named parameters, so parameter names are not significant. Some optional parameters are best explained in terms of their presence or absence, rather than through default values. In such cases, I indicate that a parameter is optional by enclosing it in brackets ([]). When more than one argument is optional, the brackets are nested. Typographic Conventions Italic Used for filenames, program names, URLs, and to introduce new terms. Also used for Unix commands and their options. Constant width Used for all code examples, as well as for all items that appear in code, including keywords, methods, functions, classes, and modules. xii | Preface www.it-ebooks.info Constant width italic Used to show text that can be replaced with user-supplied values in code examples. Constant width bold Used for commands that must be typed on the command line, and occasionally for emphasis in code examples or to indicate code output. Using Code Examples This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact the publisher for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Python in a Nutshell, Second Edition, by Alex Martelli. Copyright 2006 O’Reilly Media, Inc., 0-596-10046-9.” How to Contact Us I have tested and verified the information in this book to the best of my ability, but you may find that features have changed (or even that I have made mistakes!). Please let the publisher know about any errors you find, as well as your suggestions for future editions, by writing to: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-928-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) There is a web page for this book, which lists errata, examples, and any additional information. You can access this page at: http://www.oreilly.com/catalog/pythonian2 To ask technical questions or comment on the book, send email to: [email protected] For more information about books, conferences, resource centers, and the O’Reilly Network, see the O’Reilly web site at: http://www.oreilly.com Preface | www.it-ebooks.info xiii Safari® Enabled When you see a Safari® Enabled icon on the cover of your favorite technology book, that means the book is available online through the O’Reilly Network Safari Bookshelf. Safari offers a solution that’s better than e-books: it’s a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com. Acknowledgments My heartfelt thanks to everybody who helped me out on this book, both in the first edition and in its current second edition. Many Python beginners, practitioners, and experts have read drafts of parts of the book and have offered feedback to help me make the book clear, precise, accurate, and readable. Out of all of them, for the quality and quantity of their feedback and other help, I must single out for special thanks my colleagues at Google, especially Neal Norwitz and Mohsin Ahmed. The first edition received indispensable help from Python experts in specific areas (Aahz on threading, Itamar Shtull-Trauring on Twisted, Mike Orr on Cheetah, Eric Jones and Paul Dubois on Numeric, and Tim Peters on threading, testing, and optimization), a wonderful group of technical reviewers (Fred Drake, Magnus Lie Hetland, Steve Holden, and Sue Giller), and the book’s editor, Paula Ferguson. The second edition benefited from the efforts of editors Jonathan Gennick and Mary O’Brien, and technical reviewers Ryan Alexander, Jeffery Collins, and Mary Gardiner. I owe special thanks to the wonderful folks in the O’Reilly Tools Group, who (both directly and personally, and through the helpful tools they developed) helped me through several difficult technical problems. As always, even though they’re back in my native Italy and my career with Google has taken me to California, my thoughts go to my family: my children Flavia and Lucio, my sister Elisabetta, and my father Lanfranco. But the one, incredible individual to which my heart gushes out in gratitude, and more than gratitude, is my wife, Anna Martelli Ravenscroft, my co-author in the second edition of the Python Cookbook, a fellow Python Software Foundation member, and the harshest, most wonderful technical reviewer any author could possibly dream of. Besides her innumerable direct contributions to this book, Anna managed to create for me, out of thin air, enough peace, quiet, and free time over the last year (despite my wonderful and challenging responsibilities as Uber Tech Lead for Google) to make this book possible. Truly, this is her book at least as much as it is mine. xiv | Preface www.it-ebooks.info I Getting Started with Python This is the Title of the Book, eMatter Edition www.it-ebooks.info Copyright © 2011 O’Reilly & Associates, Inc. All rights reserved. www.it-ebooks.info Chapter 1Introduction 1 Introduction to Python Python is a general-purpose programming language. It has been around for quite a while: Guido van Rossum, Python’s creator, started developing Python back in 1990. This stable and mature language is very high-level, dynamic, objectoriented, and cross-platform—all characteristics that are very attractive to developers. Python runs on all major hardware platforms and operating systems, so it doesn’t constrain your platform choices. Python offers high productivity for all phases of the software life cycle: analysis, design, prototyping, coding, testing, debugging, tuning, documentation, deployment, and, of course, maintenance. Python’s popularity has seen steady, unflagging growth over the years. Today, familiarity with Python is an advantage for every programmer, as Python has infiltrated every niche and has useful roles to play as a part of any software solution. Python provides a unique mix of elegance, simplicity, practicality, and power. You’ll quickly become productive with Python, thanks to its consistency and regularity, its rich standard library, and the many third-party modules that are readily available for it. Python is easy to learn, so it is quite suitable if you are new to programming, yet at the same time, it is powerful enough for the most sophisticated expert. The Python Language The Python language, while not minimalist, is rather spare for good pragmatic reasons. Once a language offers one good way to express a design idea, adding other ways has only modest benefits, while the cost in terms of language complexity grows more than linearly with the number of features. A complicated language is harder to learn and master (and implement efficiently and without bugs) than a simpler one. Any complications and quirks in a language hamper productivity in software maintenance, particularly in large projects, where many developers cooperate and often maintain code originally written by others. 3 www.it-ebooks.info Python is simple, but not simplistic. It adheres to the idea that if a language behaves a certain way in some contexts, it should ideally work similarly in all contexts. Python also follows the principle that a language should not have “convenient” shortcuts, special cases, ad hoc exceptions, overly subtle distinctions, or mysterious and tricky under-the-covers optimizations. A good language, like any other designed artifact, must balance such general principles with taste, common sense, and a high degree of practicality. Python is a general-purpose programming language, so Python’s traits are useful in just about any area of software development. There is no area where Python cannot be part of an optimal solution. “Part” is an important word here; while many developers find that Python fills all of their needs, Python does not have to stand alone. Python programs can easily cooperate with a variety of other software components, making it an ideal language for gluing together components written in other languages. Python is a very-high-level language (VHLL). This means that Python uses a higher level of abstraction, conceptually farther from the underlying machine, than do classic compiled languages such as C, C++, and Fortran, which are traditionally called high-level languages. Python is also simpler, faster to process, and more regular than classic high-level languages. This affords high programmer productivity and makes Python an attractive development tool. Good compilers for classic compiled languages can often generate binary machine code that runs much faster than Python code. However, in most cases, the performance of Python-coded applications proves sufficient. When it doesn’t, you can apply the optimization techniques covered in “Optimization” on page 474 to enhance your program’s performance while keeping the benefits of high programming productivity. Newer languages such as Java and C# are slightly higher-level (farther from the machine) than classic ones such as C and Fortran, and share some characteristics of classic languages (such as the need to use declarations) as well as some of VHLLs like Python (such as the use of portable bytecode as the compilation target in typical implementations, and garbage collection to relieve programmers from the need to manage memory). If you find you are more productive with Java or C# than with C or Fortran, try Python (possibly in the Jython or IronPython implementations, covered in “Python Implementations” on page 5) and become even more productive. In terms of language level, Python is comparable to other powerful VHLLs like Perl or Ruby. The advantages of simplicity and regularity, however, remain on Python’s side. Python is an object-oriented programming language, but it lets you develop code using both object-oriented and traditional procedural styles, and a touch of the functional programming style, too, mixing and matching as your application requires. Python’s object-oriented features are like those of C++, although they are much simpler to use. 4 | Chapter 1: Introduction to Python www.it-ebooks.info