Tài liệu Writing idiomatic python

Writing Idiomatic Python Jeff Knupp 2013 i ii Copyright 2013 by Jeff Knupp All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical means without permission in writing from the author. Jeff Knupp Visit me at www.jeffknupp.com Preface There’s a famous old quote about writing maintainable software: Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. --John Woods comp.lang.c++ While I’m not usually one for aphorisms, this one strikes a chord with me. Maybe it’s because I’ve spent my professional career writing software at huge companies, but I have yet to inherit code that didn’t eventually cause me to curse the original author at some point. Everyone (besides you, of course, dear reader) struggles to write code that’s easy to maintain. When Python became popular, many thought that, because of its terseness, it would naturally lead to more maintainable software. Alas, maintainability is not an emergent property of using an expressive language. Badly written Python code is just as unmaintainable as badly written C++, Perl, Java and all the rest of the languages known for their, ahem, readability. Terse code is not a free lunch. So what do we do? Resign ourselves to maintaining code we can’t understand? Rant on Twitter and The Daily WTF about the awful code we have to work on? What must we do to stop the pain? Write. Idiomatic. Code. It’s that simple. Idioms in a programming language are a sort of lingua franca to let future readers know exactly what we’re trying to accomplish. We may document our code extensively, write exhaustive unit tests, and hold code reviews three times a day, but the fact remains: when someone else needs to make changes, the code is king. If that someone is you, all the documentation in the world won’t help you understand unreadable code. After all, how can you even be sure the code is doing what the documentation says? We’re usually reading someone else’s code because there’s a problem. But idiomatic code helps here, too. Even if it’s wrong, when code is written idiomatiii iv PREFACE ically, it’s far easier spot bugs. Idiomatic code reduces the cognitive load on the reader. After learning a language’s idioms, you’ll spend less time wondering “Wait, why are they using a named tuple there” and more time understanding what the code actually does. After you learn and internalize a language’s idioms, reading the code of a likeminded developer feels like speed reading. You’re no longer stopping at every line, trying to figure out what it does while struggling to keep in mind what came before. Instead, you’ll find yourself almost skimming the code, thinking things like ‘OK, open a file, transform the contents to a sorted list, generate the giant report in a thread-safe way.’ When you have that level of insight into code someone else wrote, there’s no bug you can’t fix and no enhancement you can’t make. All of this sounds great, right? There’s only one catch: you have to know and use a language’s idioms to benefit. Enter Writing Idiomatic Python. What started as a hasty blog post of idioms (fueled largely by my frustration while fixing the code of experienced developers new to Python) is now a full-fledged eBook. I hope you find the book useful. It is meant to be a living document, updated in near-real time with corrections, clarifications, and additions. If you find an error in the text or have difficulty deciphering a passage, please feel free to email me at jeff@jeffknupp.com. With their permission, I’ll be adding the names of all who contribute bug fixes and clarifications to the appendix. Cheers, Jeff Knupp January, 2013 Change List Version 1.1, February 2, 2013 • New idiom: “Use sys.exit in your script to return proper error codes” idiom • Greatly expanded discussion in “Avoid comparing directly to True, False, or None” and added mention of comparison to None when checking if optional arguments were set (to match the idiom “Avoid using ”, [], and {} as default parameters to functions”. • Expanded “Use the * operator to represent the”rest” of a list” idiom expanded with additional cases • Fixed page numbering issue causing numbers in table of contents and index not to match the text • Fixed various typos and grammatical errors • Changed font size and various layout issues (some of which caused text to run off the page • Changed preface text Version 1.2, February 17, 2013 • Improved formatting for epub and Kindle versions • Fixed various typos and grammatical errors v vi CONTENTS Contents Preface Change List Version 1.1, February 2, 2013 . . . . . . . . . . . . . . . . . . . . . . . . Version 1.2, February 17, 2013 . . . . . . . . . . . . . . . . . . . . . . . Contents 1 Control Structures and Functions 1.1 If Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Avoid comparing directly to True, False, or None . . . . . 1.1.2 Avoid repeating variable name in compound if statement . 1.1.3 Avoid placing conditional branch code on the same line as the colon . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 For loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.1 Use the enumerate function in loops instead of creating an “index” variable . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Use the in keyword to iterate over an iterable . . . . . . 1.2.3 Use else to execute code after a for loop concludes . . . . 1.3 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 Avoid using '', [], and {} as default parameters to functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 Use *args and **kwargs to accept arbitrary arguments . . 2 Working with Data 2.1 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Use a list comprehension to create a transformed version of an existing list . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Use the * operator to represent the “rest” of a list . . . . iii v v v vi 1 1 1 4 5 6 6 7 8 9 9 11 15 15 15 16 CONTENTS 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Use the default parameter of dict.get to provide default values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Use a dict comprehension to build a dict clearly and efficiently . . . . . . . . . . . . . . . . . . . . . . . . . . . . Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Prefer the format function for formatting strings . . . . . . 2.3.2 Use ''.join when creating a single string for list elements 2.3.3 Chain string functions to make a simple series of transformations more clear . . . . . . . . . . . . . . . . . . . . . . . Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Use underscores in function and variable names to help mark “private” data . . . . . . . . . . . . . . . . . . . . . . . . . 2.4.2 Define __str__ in a class to show a human-readable representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Use sets to eliminate duplicate entries from Iterable containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Use a set comprehension to generate sets concisely . . . . 2.5.3 Understand and use the mathematical set operations . . . Generators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6.1 Use a generator to lazily load infinite sequences . . . . . . 2.6.2 Prefer a generator expression to a list comprehension for simple iteration . . . . . . . . . . . . . . . . . . . . . . . Context Managers . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Use a context manager to ensure resources are properly managed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8.1 Use tuples to unpack data . . . . . . . . . . . . . . . . . . 2.8.2 Use _ as a placeholder for data in a tuple that should be ignored . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9.1 Avoid using a temporary variable when performing a swap of two values . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Organizing Your Code 3.1 Modules and Packages . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Use modules for encapsulation where other languages would use Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 17 17 18 19 19 20 21 22 22 25 26 26 28 29 31 31 33 34 34 35 35 36 37 37 39 39 39 viii CONTENTS 3.2 3.3 3.4 Formatting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Use all capital letters when declaring global constant values 3.2.2 Avoid placing multiple statements on a single line . . . . . . 3.2.3 Format your code according to PEP8 . . . . . . . . . . . . . Executable Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Use sys.exit in your script to return proper error codes . 3.3.2 Use the if __name__ == '__main__' pattern to allow a file to be both imported and run directly . . . . . . . . . . Imports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Prefer absolute imports to relative imports . . . . . . 3.4.2 Do not use from foo import * to import the contents of a module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Arrange your import statements in a standard order . . . . 41 41 42 43 44 44 46 47 47 48 49 4 General Advice 4.1 Avoid Reinventing the Wheel . . . . . . . . . . . . . . . . . . . . . 4.1.1 Learn the Contents of the Python Standard Library . . . . 4.1.2 Get to know PyPI (the Python Package Index) . . . . . . . 4.2 Modules of Note . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Learn the contents of the itertools module . . . . . . . . . . 4.2.2 Use functions in the os.path module when working with directory paths . . . . . . . . . . . . . . . . . . . . . . . . . 51 51 51 52 53 53 5 Contributors 55 54 Chapter 1 Control Structures and Functions 1.1 If Statements 1.1.1 Avoid comparing directly to True, False, or None For any object, be it a built-in or user defined, there is a “truthiness” associated with the object. When checking if a condition is true, prefer relying on the implicit “truthiness” of the object in the conditional statement. The rules regarding “truthiness” are reasonably straightforward. All of the following are considered False: • • • • • • None False zero for numeric types empty sequences empty dictionaries a value of 0 or False returned when either __len__ or __nonzero__ is called Everything else is considered True (and thus most things are implicitly True). The last condition for determining False, by checking the value returned by __len__ or __nonzero__, allows you to define how “truthiness” should work for any class you create. if statements in Python make use of “truthiness” implicitly, and you should too. Instead of checking if a variable foo is True like this if foo == True: 1 2 CHAPTER 1. CONTROL STRUCTURES AND FUNCTIONS you should simply check if foo:. There are a number of reasons for this. The most obvious is that if your code changes and foo becomes an int instead of True or False, your if statement still works. But at a deeper level, the reasoning is based on the difference between equality and identity. Using == determines if two objects have the same value (as defined by their _eq attribute). Using is determines if the two objects are actually the same object. Note that while there are cases where is works as if it were comparing for equality, these are special cases and shouldn’t be relied upon. As a consequence, avoid comparing directly to False and None and empty sequences like [], {}, and (). If a list named my_list is empty, calling if my_list: will evaluate to False. There are times, however, when comparing directly to None is not just recommended, but required. A function checking if an argument whose default value is None was actually set must compare directly to None like so: def insert_value(value, position=None): """Inserts a value into my container, optionally at the specified position""" if position is not None: ... What’s wrong with if position:? Well, if someone wanted to insert into position 0, the function would act as if position hadn’t been set, since 0 evaluates to False. Note the use of is not: comparisons against None (a singleton in Python) should always use is or is not, not == (from PEP8). Just let Python’s “truthiness” do the work for you. 1.1.1.1 Harmful def number_of_evil_robots_attacking(): return 10 def should_raise_shields(): # "We only raise Shields when one or more giant robots attack, # so I can just return that value..." return number_of_evil_robots_attacking() if should_raise_shields() == True: raise_shields() print('Shields raised') 1.1. IF STATEMENTS 3 else: print('Safe! No giant robots attacking') 1.1.1.2 Idiomatic def number_of_evil_robots_attacking(): return 10 def should_raise_shields(): # "We only raise Shields when one or more giant robots attack, # so I can just return that value..." return number_of_evil_robots_attacking() if should_raise_shields(): raise_shields() print('Shields raised') else: print('Safe! No giant robots attacking') 4 1.1.2 CHAPTER 1. CONTROL STRUCTURES AND FUNCTIONS Avoid repeating variable name in compound if statement When one wants to check a variable against a number of values, repeatedly listing the variable being checked is unnecessarily verbose. Using an iterable makes the code more clear and improves readability. 1.1.2.1 Harmful is_generic_name = False name = 'Tom' if name == 'Tom' or name == 'Dick' or name == 'Harry': is_generic_name = True 1.1.2.2 Idiomatic name = 'Tom' is_generic_name = name in ('Tom', 'Dick', 'Harry') 1.1. IF STATEMENTS 5 1.1.3 Avoid placing conditional branch code on the same line as the colon Using indentation to indicate scope (like you already do everywhere else in Python) makes it easy to determine what will be executed as part of a conditional statement. if, elif, and else statements should always be on their own line. No code should follow the :. 1.1.3.1 Harmful name = 'Jeff' address = 'New York, NY' if name: print(name) print(address) 1.1.3.2 Idiomatic name = 'Jeff' address = 'New York, NY' if name: print(name) print(address) 6 CHAPTER 1. CONTROL STRUCTURES AND FUNCTIONS 1.2 For loops 1.2.1 Use the enumerate function in loops instead of creating an “index” variable Programmers coming from other languages are used to explicitly declaring a variable to track the index of a container in a loop. For example, in C++: for (int i=0; i < container.size(); ++i) { // Do stuff } In Python, the enumerate built-in function handles this role. 1.2.1.1 Harmful my_container = ['Larry', 'Moe', 'Curly'] index = 0 for element in my_container: print ('{} {}'.format(index, element)) index += 1 1.2.1.2 Idiomatic my_container = ['Larry', 'Moe', 'Curly'] for index, element in enumerate(my_container): print ('{} {}'.format(index, element)) 1.2. FOR LOOPS 7 1.2.2 Use the in keyword to iterate over an iterable Programmers coming from languages lacking a for_each style construct are used to iterating over a container by accessing elements via index. Python’s in keyword handles this gracefully. 1.2.2.1 Harmful my_list = ['Larry', 'Moe', 'Curly'] index = 0 while index < len(my_list): print (my_list[index]) index += 1 1.2.2.2 Idiomatic my_list = ['Larry', 'Moe', 'Curly'] for element in my_list: print (element) 8 1.2.3 CHAPTER 1. CONTROL STRUCTURES AND FUNCTIONS Use else to execute code after a for loop concludes One of the lesser known facts about Python’s for loop is that it can include an else clause. The else clause is executed after the iterator is exhausted, unless the loop was ended prematurely due to a break statement. This allows you to check for a condition in a for loop, break if the condition holds for an element, else take some action if the condition did not hold for any of the elements being looped over. This obviates the need for conditional flags in a loop solely used to determine if some condition held. In the scenario below, we are running a report to check if any of the email addresses our users registered are malformed (users can register multiple addresses). The idiomatic version is more concise thanks to not having to deal with the has_malformed_email_address flag. What’s more, even if another programmer wasn’t familiar with the for ... else idiom, our code is clear enough to teach them. 1.2.3.1 Harmful for user in get_all_users(): has_malformed_email_address = False print ('Checking {}'.format(user)) for email_address in user.get_all_email_addresses(): if email_is_malformed(email_address): has_malformed_email_address = True print ('Has a malformed email address!') break if not has_malformed_email_address: print ('All email addresses are valid!') 1.2.3.2 Idiomatic for user in get_all_users(): print ('Checking {}'.format(user)) for email_address in user.get_all_email_addresses(): if email_is_malformed(email_address): print ('Has a malformed email address!') break else: print ('All email addresses are valid!') 1.3. FUNCTIONS 9 1.3 Functions 1.3.1 Avoid using '', [], and {} as default parameters to functions Though this is explicitly mentioned in the Python tutorial, it nevertheless surprises even experienced developers. In short: prefer names=None to names=[] for default parameters to functions. Below is the Python Tutorial’s treatment of the issue. 1.3.1.1 Harmful # The default value [of a function] is evaluated only once. # This makes a difference when the default is a mutable object # such as a list, dictionary, or instances of most classes. For # example, the following function accumulates the arguments # passed to it on subsequent calls. def f(a, L=[]): L.append(a) return L print(f(1)) print(f(2)) print(f(3)) # This will print # # [1] # [1, 2] # [1, 2, 3] 1.3.1.2 Idiomatic # If you # calls, def f(a, if L don't want the default to be shared between subsequent you can write the function like this instead: L=None): is None: L = [] L.append(a) return L print(f(1)) print(f(2)) print(f(3)) # This will print 10 # [1] # [2] # [3] CHAPTER 1. CONTROL STRUCTURES AND FUNCTIONS 1.3. FUNCTIONS 11 1.3.2 Use *args and **kwargs to accept arbitrary arguments Oftentimes, functions need to accept an arbitrary list of positional parameters and/or keyword parameters, use a subset of them, and forward the rest to another function. Using *args and **kwargs as parameters allows a function to accept an arbitrary list of positional and keyword arguments, respectively. The idiom is also useful when maintaining backwards compatibility in an API. If our function accepts arbitrary arguments, we are free to add new arguments in a new version while not breaking existing code using fewer arguments. As long as everything is properly documented, the “actual” parameters of a function are not of much consequence. 1.3.2.1 Harmful def make_api_call(foo, bar, baz): if baz in ('Unicorn', 'Oven', 'New York'): return foo(bar) else: return bar(foo) # I need to add another parameter to `make_api_call` # without breaking everyone's existing code. # I have two options... def so_many_options(): # I can tack on new parameters, but only if I make # all of them optional... def make_api_call(foo, bar, baz, qux=None, foo_polarity=None, baz_coefficient=None, quux_capacitor=None, bar_has_hopped=None, true=None, false=None, file_not_found=None): # ... and so on ad infinitum return file_not_found def version_graveyard(): # ... or I can create a new function each time the signature # changes. def make_api_call_v2(foo, bar, baz, qux): return make_api_call(foo, bar, baz) - qux def make_api_call_v3(foo, bar, baz, qux, foo_polarity): if foo_polarity != 'reversed': return make_api_call_v2(foo, bar, baz, qux) return None 12 CHAPTER 1. CONTROL STRUCTURES AND FUNCTIONS def make_api_call_v4( foo, bar, baz, qux, foo_polarity, baz_coefficient): return make_api_call_v3( foo, bar, baz, qux, foo_polarity) * baz_coefficient def make_api_call_v5( foo, bar, baz, qux, foo_polarity, baz_coefficient, quux_capacitor): # I don't need 'foo', 'bar', or 'baz' anymore, but I have to # keep supporting them... return baz_coefficient * quux_capacitor def make_api_call_v6( foo, bar, baz, qux, foo_polarity, baz_coefficient, quux_capacitor, bar_has_hopped): if bar_has_hopped: baz_coefficient *= -1 return make_api_call_v5(foo, bar, baz, qux, foo_polarity, baz_coefficient, quux_capacitor) def make_api_call_v7( foo, bar, baz, qux, foo_polarity, baz_coefficient, quux_capacitor, bar_has_hopped, true): return true def make_api_call_v8( foo, bar, baz, qux, foo_polarity, baz_coefficient, quux_capacitor, bar_has_hopped, true, false): return false def make_api_call_v9( foo, bar, baz, qux, foo_polarity, baz_coefficient, quux_capacitor, bar_has_hopped, true, false, file_not_found): return file_not_found 1.3.2.2 Idiomatic def make_api_call(foo, bar, baz): if baz in ('Unicorn', 'Oven', 'New York'): return foo(bar) else: return bar(foo) # I need to add another parameter to `make_api_call`