第二十~章 Functions and Generators

第二十章 Comprehensions and Generations

1.List Comprehensions and Functional Tools

1.1 List Comprehensions Versus map

>>> res = list(map(ord, 'spam')) # Apply function to sequence (or other)>>> res[115, 112, 97, 109]>>> res = [ord(x) for x in 'spam'] # Apply expression to sequence (or other)>>> res[115, 112, 97, 109]

>>> [x for x in range(5) if x % 2 == 0][0, 2, 4] >>> list(filter((lambda x: x % 2 == 0), range(5)))[0, 2, 4]

>>> [x ** 2 for x in range(10) if x % 2 == 0][0, 4, 16, 36, 64]>>> list( map((lambda x: x**2), filter((lambda x: x % 2 == 0), range(10))) )[0, 4, 16, 36, 64]

[ expression for target1 in iterable1 if condition1
for target2 in iterable2 if condition2 ...
for targetN in iterableN if conditionN ]

This same syntax is inherited by set and dictionary comprehensions as well as the generator expressions

>>> M = [[1, 2, 3],[4, 5, 6],[7, 8, 9]]>>> [col + 10 for row in M for col in row] # Assign to M to retain new value[11, 12, 13, 14, 15, 16, 17, 18, 19]>>> [[col + 10 for col in row] for row in M][[11, 12, 13], [14, 15, 16], [17, 18, 19]]

2.Generator Functions and Expressions

• Generator functions (available since 2.3) are coded as normal def statements, but
  use yield statements to return results one at a time, suspending and resuming their
  state between each.
• Generator expressions (available since 2.4) are similar to the list comprehensions
  of the prior section, but they return an object that produces results on demand
  instead of building a result list.
  

2.1 Generator Functions: yield Versus return

Generator functions are compiled specially into
an object that supports the iteration protocol. And when called, they don’t return a
result: they return a result generator that can appear in any iteration context.

>>> def gensquares(N):for i in range(N):yield i ** 2>>> x = gensquares(4)>>> x<generator object gensquares at 0x0000000003284480>>>> next(x)0>>> next(x)1>>> next(x)4>>> x.__next__()9>>> x.__next__()Traceback (most recent call last):  File "<pyshell#88>", line 1, in <module>    x.__next__()StopIteration>>> y = gensquares(5) # Returns a generator which is its own iterator>>> iter(y) is y # iter() is not required: a no-op hereTrue

generators can be better in terms of both memory use and performance in larger programs

generators can provide a simpler alternative to manually saving the state between iterations in class objects

Extended generator function protocol: send versus next

>>> def gen():        for i in range(10):            X = yield i            print(X)>>> G = gen()>>> next(G) # Must call next() first, to start generator0>>> G.send(77) # Advance, and send value to yield expression771>>> G.send(88)882>>> next(G) # next() and X.__next__() send NoneNone3

Generator Expressions: Iterables Meet Comprehensions

>>> [x ** 2 for x in range(4)] # List comprehension: build a list[0, 1, 4, 9]>>> (x ** 2 for x in range(4)) # Generator expression: make an iterable<generator object <genexpr> at 0x00000000029A8288>>>> list(x ** 2 for x in range(4)) # List comprehension equivalence[0, 1, 4, 9]

>>> G = (x ** 2 for x in range(4))>>> iter(G) is G # iter(G) optional: __iter__ returns selfTrue>>> next(G) # Generator objects: automatic methods0>>> next(G)1>>> next(G)4>>> next(G)9>>> next(G)Traceback (most recent call last):File "<stdin>", line 1, in <module>StopIteration>>> G<generator object <genexpr> at 0x00000000029A8318>

>>> for num in (x ** 2 for x in range(4)): # Calls next() automaticallyprint('%s, %s' % (num, num / 2.0))0, 0.01, 0.54, 2.09, 4.5

>>> ''.join(x.upper() for x in 'aaa,bbb,ccc'.split(','))  #doesn’t require extra parentheses around the generator.'AAABBBCCC'                                       #parentheses are not required around a generator expression                                                    #that is the sole item already enclosed in parentheses used for other purposes                                                  #like those of a function call.>>> sum(x ** 2 for x in range(4)) # Parens optional14>>> sorted(x ** 2 for x in range(4)) # Parens optional[0, 1, 4, 9]>>> sorted((x ** 2 for x in range(4)), reverse=True) # Parens required[9, 4, 1, 0]

>>> line = 'aaa,bbb,ccc'>>> ''.join([x.upper() for x in line.split(',')]) # Makes a pointless list'AAABBBCCC'>>> ''.join(x.upper() for x in line.split(',')) # Generates results'AAABBBCCC'>>> ''.join(map(str.upper, line.split(','))) # Generates results'AAABBBCCC'

>>> [x * 2 for x in [abs(x) for x in (−1, −2, 3, 4)]] # Nested comprehensions[2, 4, 6, 8]>>> list(map(lambda x: x * 2, map(abs, (−1, −2, 3, 4)))) # Nested maps[2, 4, 6, 8]>>> list(x * 2 for x in (abs(x) for x in (−1, −2, 3, 4))) # Nested generators[2, 4, 6, 8]

Generator expressions versus filter

>>> line = 'aa bbb c'>>> ''.join(x for x in line.split() if len(x) > 1) # Generator with 'if''aabbb'>>> ''.join(filter(lambda x: len(x) > 1, line.split())) # Similar to filter'aabbb'

Generator Functions Versus Generator Expressions

• Generator functions
  A function def statement that contains a yield statement is turned into a generator
  function. When called, it returns a new generator object with automatic retention
  of local scope and code position; an automatically created __iter__ method that
  simply returns itself; and an automatically created __next__ method (next in 2.X)
  that starts the function or resumes it where it last left off, and raises StopItera
  tion when finished producing results.
• Generator expressions
  A comprehension expression enclosed in parentheses is known as a generator expression.
  When run, it returns a new generator object with the same automatically
  created method interface and state retentionas a generator function call’s results
  —with an __iter__ method that simply returns itself; and a _next__ method
  (next in 2.X) that starts the implied loop or resumes it where it last left off, and
  raises StopIteration when finished producing results.
  

Generators Are Single-Iteration Objects

>>> G = (c * 4 for c in 'SPAM')>>> iter(G) is G # My iterator is myself: G has __next__True

如果重头开始，重新生成  (c * 4 for c in 'SPAM')对象

>>> def both(N):        for i in range(N): yield i        for i in (x ** 2 for x in range(N)): yield i>>> list(both(5))[0, 1, 2, 3, 4, 0, 1, 4, 9, 16]>>> def both(N):        yield from range(N)           #3.3 syntax        yield from (x ** 2 for x in range(N))>>> list(both(5))[0, 1, 2, 3, 4, 0, 1, 4, 9, 16]

>>> import os>>> for (root, subs, files) in os.walk('.'): # Directory walk generator        for name in files: # A Python 'find' operation            if name.startswith('call'):                print(root, name)                . callables.py.\dualpkg callables.py

Scopes and Comprehension Variables

3.X variables assigned in a comprehension are really a

further nested special-case scope; other names referenced within these expressions follow
the usual LEGB rules.

c:\code> py −3>>> (X for X in range(5))<generator object <genexpr> at 0x00000000028E4798>>>> XNameError: name 'X' is not defined>>> X = 99>>> [X for X in range(5)] # 3.X: generator, set, dict, and list localize[0, 1, 2, 3, 4]>>> X99 >>> Y = 99>>> for Y in range(5): pass # But loop statements do not localize names>>> Y4

>>> {x: y for x in [1, 2, 3] for y in [4, 5, 6]} # Neither do dict keys 此处更新了值{1: 6, 2: 6, 3: 6}

第二十一章 The Benchmarking Interlude

timeit

Function Gotchas：

• Local Names Are Detected Statically
• Defaults and Mutable Objects
• Functions Without returns：Technically, all functions return a value; if you don’t provide a return
  statement, your function returns the None object automatically