<the hacker guide to python>读书笔记

1.Coding style & automated checks

And so the Python community, in their vast wisdom, came up with the PEP8 standard for writing Python code. The list of guidelines boils down to:
• Use 4 spaces per indentation level.
• Limit all lines to a maximum of 79 characters.
• Separate top-level function and class definitions with two blank lines.
• Encode files using ASCII or UTF-8.
• One module import per import statement and per line, at the top of the file, after comments and docstrings, grouped first b⁴ standard, then third-party, and finally
local library imports.
• No extraneous whitespaces between parentheses, brackets, or braces, or before commas.

• Name classes in CamelCase; suffix exceptions with Error (if applicable); name functions in lowercase with words separated_by_underscores; and use a leading underscore
for _private attributes or methods.

In its quest for beautiful and unified code, the OpenStack project chose flake8 for all of its code checks. However, as time passed, the hackers took advantage of flake8's
extensibility to test for even more potential issues with submitted code. The end result of all this is a flake8 extension called hacking. It checks for errors such as odd usage of except, Python 2/3 portability issues, import style, dangerous string formatting, and possible localization issues.

2.Modules and libraries

Python Enhancement Proposal (PEP)

warnings module and deprecate

3.Documentation

reStructuredText, or reST for short

sphinx

4 Distribution

pbr stands for Python Build Reasonableness. The project has been started inside OpenStack as a set of tools around setuptools to facilitate installation and deployment of packages. It takes inspiration from distutils2, using a setup.cfg file to describe the packager’s intents.

5 Virtual Environment

The venv module is now part of Python 3.3 and above, and allows to handle virtual environment without using the virtualenv package or any other one

The downside of this venv module is that it doesn’t install setuptools nor pip by default.

6 Unit testing

Your tests should be stored inside a tests submodule of your application or library.

It’s usually simpler to use a hierarchy in your test tree that mimics the hierarchy you have in your module tree. This means that the tests covering the code of mylib/foobar.py should be inside mylib/tests/test_foobar.py; this makes things simpler when looking for the tests relating to a particular file.

the nose package comes to the rescue – once installed, it provides thenosetests command, which loads every file whose name starts with test_
and then executes all functions within that start with test_.

unittest is part of the Python standard library.

6.2 fixtures

6.3 mock

6.4 testscenarios

6.5 subunit is a Python module that provides a streaming protocol for test results

6.6 Code coverage is a tool which complements unit testing. It uses code analysis tools and tracing hooks to determine which lines of your code have been executed; when used during a unit test run, it can show you which parts of your code base have been crossed over and which parts have not.

6.7 Tox aims to automate and standardize how tests are run in Python. To that end,it provides everything needed to run an entire test suite in a clean virtual environment,while also installing your application to check that the installation works fine.

Before using tox, you need to provide a configuration file. This file is named tox.ini and should be placed in the root directory of your project, beside your setup.py file.

7 Methods and decorators

7.1 creating decorators

a decorator replaces the original function with a new one built on-the-fly. However, this new function lacks many of the attributes of the original function, such as its docstring and its name

the functools module included in Python solves this problem with the update_wrapper function, which copies these attributes to the wrapper itself.

>>> def foobar(username="someone"):
... """Do crazy stuff."""
... pass
...
>>> foobar = functools.update_wrapper(is_admin, foobar)
>>> foobar.__name__
'foobar'
>>> foobar.__doc__
'Do crazy stuff.'

It can get tedious to use update_wrapper manually when creating decorators, so functools provides a decorator for decorators called wraps

import functools
def check_is_admin(f):
@functools.wraps(f)
def wrapper(*args, **kwargs):
if kwargs.get('username') != 'admin':
raise Exception("This user is not allowed to get food")
return f(*args, **kwargs)
return wrapper

7.2 How methods work in Python

>>> m = Pizza(42).get_size
>>> m()
42

上面要注意：函数和函数的调用

7.3 Static methods

当一个函数逻辑上属于一个类又不依赖与类的属性的时候，可以使用 staticmethod

7.4 Class method

Class methods are mostly useful for creating factory methods – methods which instantiate objects in a specific fashion.

7.5 Abstract methods

If you implement your abstract methods using Python’s built-in abc module instead, you’ll get an early warning if you try to instantiate an object with abstract methods:

7.6 Mixing static, class, and abstract methods

8 Functional programming

8.1 Generators

You can check whether a function is a generator or not yourself by using inspect.isgeneratorfunction

Python 3 provides another useful function, inspect.getgeneratorstate

yield also has a less commonly used feature: it can return a value like a function call. This allows us to pass a value to a generator by calling its send() method:

8.2 List comprehensions

List comprehension, or listcomp for short, allows you to define a list’s contents inline with its declaration

8.3 Functional functions functioning

Python includes a number of tools for functional programming. These built-in functions cover the basics:

map(function, iterable) applies function to each item in iterable and returns either a list in Python 2 or an iterable map object in Python 3

filter(function or None, iterable) filters the items in iterable based on the result returned by function, and returns either a list in Python 2, or better, an
iterable filter object in Python 3

enumerate(iterable[, start]) returns an iterable enumerate object that yields a sequence of tuples, each consisting of an integer index (starting with start, if
provided) and the corresponding item in iterable

sorted(iterable, key=None, reverse=False) returns a sorted version of iterable. The key argument allows you to provide a function that returns the value to
sort on.

any(iterable) and all(iterable) both return a boolean depending on the values returned by iterable

zip(iter1 [,iter2 [...]]) takes multiple sequences and combines them into tuples.

If you’re planning to write code using these functions, keep in mind that you’ll get the most benefit out of them in Python 3. If you’re stuck to Python 2,
don’t despair yet: the itertools module from the standard library provides an iterator based version of many of these functions (itertools.izip, itertoolz.imap,
itertools.ifilter, etc).

the small but very useful Python package first

functools.partial is typically useful in replacement of lambda, and is to be considered as a superior alternative.

The itertools module in the Python Standard Library also provides a bunch of useful functions that you’ll want to keep in mind. I’ve seen too many programmers end
up writing their own versions of these functions even though Python itself provides them out-of-the-box:

• chain(*iterables) iterates over multiple iterables one ater each other without building an intermediate list of all items.
• combinations(iterable, r) generates all combination of length r from the given iterable.
• compress(data, selectors) applies a boolean mask from selectors to data and returns only the values from data where the corresponding element of selectors
is true.
• count(start, step) generates an endless sequence of values, starting from start and incrementing by step with each call.
• cycle(iterable) loops repeatedly over the values in iterable.
• dropwhile(predicate, iterable) filters elements of an iterable starting from the beginning until predicate is false.
• groupby(iterable, keyfunc) creates an iterator grouping items by the result returned by the keyfunc function.

• permutations(iterable[, r]) returns successive r-length permutations of the items in iterable.
• product(*iterables) returns an iterable of the cartesian product of iterables without using a nested for loop.
• takewhile(predicate, iterable) returns elements of an iterable starting from the beginning until predicate is false.

9 The AST

AST stands for Abstract Syntax Tree. It is a tree representation of the abstract structure of the source code of any programming language, including Python

10 Performances and optimizations

10.1 Data structures

10.2 Profiling

Python provides a few tools to profile your program. The standard one is cProfile and is easy enough to use.

10.5 Memoization

Memoization is a technique used to speed up function calls by caching their result

Starting with Python 3.3, the functools module provides a LRU (Least-Recently-Used) cache decorator.

10.7 Achieving zero copy with the buffer protocol

you can use the memoryview class constructor on it to build a new memoryview object that will reference the original object memory.

11 Scaling and architecture

11.1 A note on multi-threading

GIL story

Jython, for example, doesn’t have a global interpreter lock, which means that it can run multiple threads in parallel efficiently.

11.2 Multiprocessing vs multithreading

As explained earlier, multi-threading is not a good scalability solution because of the GIL. A better solution is the multiprocessing package that is provided with Python.

11.3 Asynchronous and event-driven architecture

Alternatively, there are many frameworks which provide this kind of functionality in a more integrated manner, such as Twisted or Tornado. Twisted has been almost a
de-facto standard for years in this regard. C libraries that export Python interfaces, such as libevent, libev or libuv, also provides very efficient event loops.

While they all solve the same problem, the downside is that nowadays there are too many choices, and most of them are not interoperable. Also,most of them are
callback based – which means that the program flow is not really clear when reading the code.
What about gevent or Greenlet? They avoid the use of callback, but the implementation details are scary, and include CPython x86 specific code and monkey-patching of standard functions

tulip has been renamed and merged into Python 3.4 as the asynciopackage.

11.4 Service-oriented architecture