Tài liệu Mastering object oriented python

Mastering Object-oriented Python Grasp the intricacies of object-oriented programming in Python in order to efficiently build powerful real-world applications Steven F. Lott BIRMINGHAM - MUMBAI Mastering Object-oriented Python Copyright © 2014 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: April 2014 Production Reference: 1150414 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK ISBN 978-1-78328-097-1 www.packtpub.com Cover Image by Duraid Fatouhi ([email protected]) Credits Author Steven F. Lott Copy Editors Insiya Morbiwala Kirti Pai Reviewers Mike Driscoll Róman Joost Sakis Kasampalis Stuti Srivastava Project Coordinator Akash Poojary Albert Lukaszewski, Ph.D Hugo Solis Commissioning Editor Usha Iyer Proofreaders Stephen Copestake Clyde Jenkins Linda Morris Jonathan Todd Acquisition Editor Gregory Wild Indexer Mariammal Chettiyar Content Development Editor Shaon Basu Graphics Abhinash Sahu Technical Editors Kapil Hemnani Monica John Production Coordinator Alwin Roy Akashdeep Kundu Cover Work Alwin Roy About the Author Steven F. Lott has been programming since the 70s, when computers were large, expensive, and rare. As a contract software developer and architect, he has worked on hundreds of projects from very small to very large. He's been using Python to solve business problems for over 10 years. Steven is currently a technomad who lives in various places on the east coast of the US. His technology blog is: http://slott-softwarearchitect.blogspot.com I owe deep gratitude to Floating Leaf for all her support and guidance. About the Reviewers Mike Driscoll has been programming in Python since 2006. He enjoys writing about Python on his blog at http://www.blog.pythonlibrary.org/. He has co-authored Core Python refcard for DZone. Mike has also been a technical reviewer for various books of Packt Publishing, such as Python 3 Object Oriented Programming, Python 2.6 Graphics Cookbook, and Tkinter GUI Application Development Hotshot. Mike recently wrote the book Python 101. I would like to thank my beautiful wife, Evangeline, for always supporting me. I would also like to thank my friends and family for all that they do to help me. I would also like to thank Jesus Christ for saving me. Róman Joost first learned about open source software in 1997. He is the project manager of GIMP's user documentation. He has contributed to GIMP and Python/ Zope open source projects for eight years. Róman works for Red Hat in Brisbane, Australia. Sakis Kasampalis is based in the Netherlands, where he currently works as a Software Engineer for a location-based B2B provider. He is not dogmatic about particular programming languages and tools; his principle is that the right tool should be used for the right job. One of his favorite tools is Python because he finds it very productive. Among the FOSS activities of Kasampalis is maintaining a GitHub repository that is related to implementing design patterns in Python, which are available at https:// github.com/faif/python-patterns. He was also a technical reviewer of the book Learning Python Design Patterns, Packt Publishing. Albert Lukaszewski, Ph.D, is principal consultant for Lukaszewski Consulting Services in southeast Scotland. Having programmed computers for over 30 years, he consults on the system design and implementation. Previously, he served as Chief Engineer for ACCESS Europe GmbH. Much of his experience is related to text processing, database systems, and Natural Language Processing (NLP). In addition to MySQL for Python, Packt Publishing, he previously wrote a column on Python for the New York Times subsidiary, About.com. Hugo Solis is an assistant professor in the Physics department at the University of Costa Rica. His current research interests are computational cosmology, complexity, and the influence of hydrogen on material properties. He has wide experience with languages including C/C++ and Python for scientific programming and visualization. He is a member of the Free Software Foundation and has contributed code to some free software projects. Currently, he is in charge of the IFT, a Costa Rican scientific, non-profit organization for the multidisciplinary practice of physics (http://iftucr.org). I'd like to thank Katty Sanchez, my beloved mother, for her support and vanguard thoughts. www.PacktPub.com Support files, eBooks, discount offers and more You might want to visit www.PacktPub.com for support files and downloads related to your book. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub. com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. TM http://PacktLib.PacktPub.com Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books.  Why Subscribe? • Fully searchable across every book published by Packt • Copy and paste, print and bookmark content • On demand and accessible via web browser Free Access for Packt account holders If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access. Table of Contents Preface 1 Some Preliminaries 9 About casino Blackjack Playing the game Blackjack player strategies Object design for simulating Blackjack Performance – the timeit module Testing – unittest and doctest Unit testing and technology spikes Docstrings – RST markup and documentation tools The IDE question About special method names Summary 10 10 11 12 12 13 15 16 17 18 19 Part 1: Pythonic Classes via Special Methods Chapter 1: The __init__() Method The implicit superclass – object The base class object __init__() method Implementing __init__() in a superclass Using __init__() to create manifest constants Leveraging __init__() via a factory function Faulty factory design and the vague else clause Simplicity and consistency using elif sequences Simplicity using mapping and class objects Two parallel mappings Mapping to a tuple of values The partial function solution Fluent APIs for factories 25 25 26 27 28 30 31 32 33 34 34 34 35 Table of Contents Implementing __init__() in each subclass 36 Simple composite objects 38 Wrapping a collection class 39 Extending a collection class 39 More requirements and another design 40 Complex composite objects 41 Complete composite object initialization 42 Stateless objects without __init__() 43 Some additional class definitions 43 Multi-strategy __init__() 46 More complex initialization alternatives 47 Initializing static methods 48 Yet more __init__() techniques 49 Initialization with type validation 52 Initialization, encapsulation, and privacy 54 Summary 55 Chapter 2: Integrating Seamlessly with Python – Basic Special Methods The __repr__() and __str__() methods Non collection __str__() and __repr__() Collection __str__() and __repr__() The __format__() method Nested formatting specifications Collections and delegating format specifications The __hash__() method Deciding what to hash Inheriting definitions for immutable objects Overriding definitions for immutable objects Overriding definitions for mutable objects Making a frozen hand from a mutable hand The __bool__() method The __bytes__() method The comparison operator methods Designing comparisons Implementation of comparison for objects of the same class Implementation of comparison for objects of mixed classes Hard totals, soft totals, and polymorphism A mixed class comparison example The __del__() method The reference count and destruction [ ii ] 57 58 59 60 61 63 64 64 65 67 69 71 72 74 75 77 80 81 83 83 85 88 88 Table of Contents Circular references and garbage collection Circular references and the weakref module The __del__() and close() methods The __new__() method and immutable objects The __new__() method and metaclasses Metaclass example 1 – ordered attributes Metaclass example 2 – self-reference Summary 90 91 93 94 96 97 99 102 Chapter 3: Attribute Access, Properties, and Descriptors 105 Chapter 4: The ABCs of Consistent Design 131 Basic attribute processing Attributes and the __init__() method Creating properties Eagerly computed properties Setter and deleter properties Using special methods for attribute access Creating immutable objects with __slots__ Creating immutable objects as a tuple subclass Eagerly computed attributes The __getattribute__() method Creating descriptors Using a nondata descriptor Using a data descriptor Summary, design considerations, and trade-offs Properties versus attributes Designing with descriptors Looking forward 106 107 108 110 112 113 114 116 117 119 121 123 125 127 128 129 129 Abstract base classes 131 Base classes and polymorphism 134 Callables 135 Containers and collections 136 Numbers 137 Some additional abstractions 138 The iterator abstraction 138 Contexts and context managers 139 The abc module 140 Summary, design considerations, and trade-offs 142 Looking forward 143 [ iii ] Table of Contents Chapter 5: Using Callables and Contexts 145 Chapter 6: Creating Containers and Collections 163 Designing with ABC callables 145 Improving performance 148 Using memoization or caching 149 Using functools for memoization 150 Aiming for simplicity using the callable API 151 Complexities and the callable API 152 Managing contexts and the with statement 154 Using the decimal context 155 Other contexts 156 Defining the __enter__() and __exit__() methods 156 Handling exceptions 158 Context manager as a factory 158 Cleaning up in a context manager 159 Summary 161 Callable design considerations and trade-offs 161 Context manager design considerations and trade-offs 162 Looking forward 162 ABCs of collections Examples of special methods Using the standard library extensions The namedtuple() function The deque class The ChainMap use case The OrderedDict collection The defaultdict subclass The counter collection Creating new kinds of collections Defining a new kind of sequence A statistical list Choosing eager versus lazy calculation Working with __getitem__(), __setitem__(), __delitem__(), and slices Implementing __getitem__(), __setitem__(), and __delitem__() Wrapping a list and delegating Creating iterators with __iter__() Creating a new kind of mapping Creating a new kind of set Some design rationale Defining the Tree class [ iv ] 164 165 165 166 168 170 172 174 175 177 178 179 180 183 184 186 188 189 191 192 193 Table of Contents Defining the TreeNode class 194 Demonstrating the binary tree set 197 Summary 198 Design considerations and Trade-offs 198 Looking forward 200 Chapter 7: Creating Numbers 201 Chapter 8: Decorators and Mixins – Cross-cutting Aspects 223 ABCs of numbers 202 Deciding which types to use 203 The method resolution and the reflected operator concept 204 The arithmetic operator's special methods 205 Creating a numeric class 208 Defining FixedPoint initialization 208 Defining FixedPoint binary arithmetic operators 210 Defining FixedPoint unary arithmetic operators 212 Implementing FixedPoint reflected operators 213 Implementing FixedPoint comparison operators 216 Computing a numeric hash 217 Designing more useful rounding 218 Implementing other special methods 219 Optimization with the in-place operators 220 Summary 221 Design considerations and trade-offs 221 Looking forward 222 Class and meaning Constructing the functions Constructing the class Some class design principles Aspect-oriented programming Using built-in decorators Using standard library decorators Using standard library mixin classes Using the context manager mixin class Turning off a class feature Writing a simple function decorator Creating separate loggers Parameterizing a decorator Creating a method function decorator Creating a class decorator [v] 224 224 226 227 227 228 230 231 231 233 234 235 236 238 240 Table of Contents Adding method functions to a class 242 Using decorators for security 243 Summary 245 Design considerations and trade-offs 245 Looking forward 246 Part 2: Persistence and Serialization Chapter 9: Serializing and Saving – JSON, YAML, Pickle, CSV, and XML Understanding persistence, class, state, and representation Common Python terminologies Filesystem and network considerations Defining classes to support persistence Rendering a blog and posts Dumping and loading with JSON Supporting JSON in our classes Customizing JSON encoding Customizing JSON decoding The security and the eval() issue Refactoring the encode function Standardizing the date string Writing JSON to a file Dumping and loading with YAML Formatting YAML data on a file Extending the YAML representation Security and safe loading Dumping and loading with pickle Designing a class for reliable pickle processing Security and the global issue Dumping and loading with CSV Dumping simple sequences to CSV Loading simple sequences from CSV Handling containers and complex classes Dumping and loading multiple row types in a CSV file Filtering CSV rows with an iterator Dumping and loading joined rows in a CSV file Dumping and loading with XML Dumping objects using string templates Dumping objects with xml.etree.ElementTree Loading XML documents [ vi ] 251 253 254 255 255 258 260 262 263 265 266 266 268 269 270 271 272 275 276 277 279 280 281 282 283 284 286 287 290 291 293 294 Table of Contents Summary 295 Design considerations and trade-offs 295 Schema evolution 297 Looking forward 297 Chapter 10: Storing and Retrieving Objects via Shelve 299 Chapter 11: Storing and Retrieving Objects via SQLite 327 Analyzing persistent object use cases 300 The ACID properties 301 Creating a shelf 302 Designing shelvable objects 303 Designing keys for our objects 303 Generating surrogate keys for objects 305 Designing a class with a simple key 305 Designing classes for containers or collections 308 Referring to objects via foreign keys 308 Designing CRUD operations for complex objects 311 Searching, scanning, and querying 312 Designing an access layer for shelve 313 Writing a demonstration script 317 Creating indexes to improve efficiency 318 Creating top-level indices 320 Adding yet more index maintenance 321 The writeback alternative to index updates 323 Schema evolution 323 Summary 325 Design considerations and trade-offs 325 Application software layers 326 Looking forward 326 SQL databases, persistence, and objects The SQL data model – rows and tables CRUD processing via SQL DML statements Querying rows with the SQL SELECT statement SQL transactions and the ACID properties Designing primary and foreign database keys Processing application data with SQL Implementing class-like processing in pure SQL Mapping Python objects to SQLite BLOB columns Mapping Python objects to database rows manually Designing an access layer for SQLite Implementing container relationships [ vii ] 328 329 331 333 335 337 339 340 342 344 346 349 Table of Contents Improving performance with indices 350 Adding an ORM layer 351 Designing ORM-friendly classes 352 Building the schema with the ORM layer 355 Manipulating objects with the ORM layer 357 Querying post objects given a tag string 359 Improving performance with indices 361 Schema evolution 361 Summary 363 Design considerations and trade-offs 363 Mapping alternatives 364 Keys and key designs 364 Application software layers 365 Looking forward 366 Chapter 12: Transmitting and Sharing Objects Class, state, and representation Using HTTP and REST to transmit objects Implementing CRUD operations via REST Implementing non-CRUD operations The REST protocol and ACID Choosing a representation – JSON, XML, or YAML Implementing a REST server – WSGI and mod_wsgi Creating a simple REST application and server Implementing a REST client Demonstrating and unit testing the RESTful services Using Callable classes for WSGI applications Designing RESTful object identifiers Multiple layers of REST services Creating the roulette server Creating the roulette client Creating a secure REST service The WSGI Authentication application Implementing REST with a web application framework Using a message queue to transmit objects Defining processes Building queues and supplying data Summary Design considerations and trade-offs Schema evolution Application software layers Looking forward [ viii ] 367 368 368 369 371 371 372 372 374 377 378 380 382 383 389 390 391 394 395 396 397 399 401 401 402 402 403 Table of Contents Chapter 13: Configuration Files and Persistence Configuration file use cases Representation, persistence, state, and usability Application configuration design patterns Configuring via object construction Implementing a configuration hierarchy Storing the configuration in the INI files Handling more literals via the eval() variants Storing the configuration in PY files Configuration via class definitions Configuration via SimpleNamespace Using Python with exec() for the configuration Why is exec() a nonproblem? Using ChainMap for defaults and overrides Storing the configuration in JSON or YAML files Using flattened JSON configurations Loading a YAML configuration Storing the configuration in property files Parsing a properties file Using a properties file Storing the configuration in XML files – PLIST and others Customized XML configuration files Summary Design considerations and trade-offs Creating a shared configuration Schema evolution Looking Forward 405 406 408 408 409 411 413 416 417 418 420 422 424 425 427 429 430 432 432 435 436 438 440 440 441 442 442 Part 3: Testing, Debugging, Deploying, and Maintaining Chapter 14: The Logging and Warning Modules Creating a basic log Creating a shared class-level logger Configuring the loggers Starting up and shutting down the logging system Naming the loggers Extending the logger levels Defining handlers for multiple destinations Managing the propagation rules Configuration gotcha [ ix ] 447 448 449 450 450 452 453 454 457 458 Table of Contents Specializing logging for control, debug, audit, and security 458 Creating a debugging log 460 Creating audit and security logs 461 Using the warnings module 463 Showing API changes with a warning 465 Showing configuration problems with a warning 466 Showing possible software problems with a warning 467 Advanced logging – the last few messages and network destinations 468 Building an automatic tail buffer 468 Sending logging messages to a remote process 471 Preventing queue overrun 475 Summary 476 Design considerations and trade-offs 476 Looking forward 477 Chapter 15: Designing for Testability 479 Chapter 16: Coping With the Command Line 511 Defining and isolating units for testing Minimizing the dependencies Creating simple unit tests Creating a test suite Including edge and corner cases Mocking dependencies for testing Using more mocks to test more behaviors Using doctest to define test cases Combining doctest and unittest Creating a more complete test package Using setup and teardown Using setup and teardown with OS resources Using setup and teardown with databases The TestCase class hierarchy Using externally defined expected results Automated integration or performance testing Summary Design considerations and trade-offs Looking forward The OS interface and the command line Arguments and options Parsing the command line with argparse A simple on/off option An option with an argument [x] 480 480 483 485 486 487 490 491 495 495 496 498 499 503 504 507 509 510 510 511 513 514 516 517 Table of Contents Positional arguments 517 All other arguments 518 --version display and exit 519 --help display and exit 519 Integrating command-line options and environment variables 519 Providing more configurable defaults 520 Overriding configuration file settings with environment variables 521 Overriding environment variables with the configuration files 522 Making the configuration aware of the None values 523 Customizing the help output 523 Creating a top-level main() function 525 Ensuring DRY for the configuration 528 Managing nested configuration contexts 528 Programming In The Large 529 Designing command classes 530 Adding the analysis command subclass 532 Adding and packaging more features into an application 533 Designing a higher-level composite command 533 Additional composite command design patterns 535 Integrating with other applications 537 Summary 538 Design considerations and trade-offs 538 Looking forward 538 Chapter 17: The Module and Package Design Designing a module Some module design patterns Module versus class The expected content of a module Whole module versus module items Designing a package Designing a module-package hybrid Designing a package with alternate implementations Designing a main script and the __main__ module Creating an executable script file Creating a __main__ module Programming in the large Designing long-running applications Organizing code into src, bin, and test Installing Python modules [ xi ] 539 540 540 542 543 545 546 547 548 550 551 552 552 553 555 557