第十四章 Iterations and Comprehensions

“iterable objects”：an object is considered iterable if it is either a physically stored sequence, or an object that produces one result at a time in the context of an iteration tool like a for loop。

iterable objects include both physical sequences and virtual sequences computed on demand.

读取文件时，可以使用文件的readline()方法，到文件结尾返回空字符串。

也可使用__next__方法，但在文件结尾时产生 built-in StopIteration exception

Any object with a __next__ method to advance to a next result, which raisesStopIterationat the end of the series of results, is considered an iterator in Python.

all iteration tools normally work internally by calling __next__ on each iteration and catching the StopIteration exception to determine when to exit.

>>> for line in open('script2.py'): # Use file iterators to read by lines... print(line.upper(), end='') # Calls __next__, catches StopIteration...IMPORT SYSPRINT(SYS.PATH)X = 2PRINT(X ** 32)  #最佳读文件方法：it’s the simplest to code, might be the quickest to run, and is the best in terms of memory usage.

Manual Iteration: iter and next：

>>> f = open('script2.py')>>> f.__next__() # Call iteration method directly'import sys\n'>>> f.__next__()'print(sys.path)\n'>>> f = open('script2.py')>>> next(f) # The next(f) built-in calls f.__next__() in 3.X'import sys\n'>>> next(f) # next(f) => [3.X: f.__next__()], [2.X: f.next()]'print(sys.path)\n'

• The iterable object you request iteration for, whose __iter__ is run by iter
• The iterator object returned by the iterable that actually produces values during the iteration, whose __next__ is run by next and raises StopIteration when finished producing results
  

>>> L = [1, 2, 3]>>> I = iter(L) # Obtain an iterator object from an iterable>>> I.__next__() # Call iterator's next to advance to next item1>>> I.__next__() # Or use I.next() in 2.X, next(I) in either line2>>> I.__next__()3>>> I.__next__()...error text omitted...StopIteration

a file object is its own iterator，support just one iteration

>>> f = open('script2.py')>>> iter(f) is fTrue>>> iter(f) is f.__iter__()True>>> f.__next__()'import sys\n'

Lists and many other built-in objects, though, are not their own iterators，support multiple open iterations

>>> L = [1, 2, 3]>>> iter(L) is LFalse>>> L.__next__()AttributeError: 'list' object has no attribute '__next__'>>> I = iter(L)>>> I.__next__()1>>> next(I) # Same as I.__next__()2

Manual iteration：

>>> L = [1, 2, 3]>>>>>> for X in L: # Automatic iteration... print(X ** 2, end=' ') # Obtains iter, calls __next__, catches exceptions...1 4 9>>> I = iter(L) # Manual iteration: what for loops usually do>>> while True:... try: # try statement catches exceptions... X = next(I) # Or call I.__next__ in 3.X... except StopIteration:... break... print(X ** 2, end=' ')...1 4 9

Other Built-in Type Iterables（shelves，the results from os.popen，enumerate）

>>> D = {'a':1, 'b':2, 'c':3}>>> for key in D.keys():... print(key, D[key])...a 1b 2c 3>>> I = iter(D)>>> next(I)'a'>>> next(I)'b'>>> next(I)'c'>>> next(I)Traceback (most recent call last):File "<stdin>", line 1, in <module>StopIteration>>> for key in D:... print(key, D[key])...a 1b 2c 3

1.List Comprehensions: A First Detailed Look

>>> L = [1, 2, 3, 4, 5]>>> for i in range(len(L)):   #原位置改变其值... L[i] += 10...>>> L[11, 12, 13, 14, 15]>>> L = [x + 10 for x in L]  #生成新L>>> L[21, 22, 23, 24, 25]

Extended List Comprehension Syntax

Filter clauses: if

>>> lines = [line.rstrip() for line in open('script2.py') if line[0] == 'p']>>> lines['print(sys.path)', 'print(x ** 32)']

Nested loops: for

>>> [x + y for x in 'abc' for y in 'lmn']['al', 'am', 'an', 'bl', 'bm', 'bn', 'cl', 'cm', 'cn']

2.Other Iteration Contexts

>>> uppers = [line.upper() for line in open('script2.py')]>>> uppers['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(X ** 32)\n']>>> map(str.upper, open('script2.py')) # map is itself an iterable in 3.X<map object at 0x00000000029476D8>>>> list(map(str.upper, open('script2.py')))['IMPORT SYS\n', 'PRINT(SYS.PATH)\n', 'X = 2\n', 'PRINT(X ** 32)\n']

>>> sorted(open('script2.py'))['import sys\n', 'print(sys.path)\n', 'print(x ** 32)\n', 'x = 2\n']>>> list(zip(open('script2.py'), open('script2.py')))[('import sys\n', 'import sys\n'), ('print(sys.path)\n', 'print(sys.path)\n'),('x = 2\n', 'x = 2\n'), ('print(x ** 32)\n', 'print(x ** 32)\n')]>>> list(enumerate(open('script2.py')))[(0, 'import sys\n'), (1, 'print(sys.path)\n'), (2, 'x = 2\n'),(3, 'print(x ** 32)\n')]>>> list(filter(bool, open('script2.py'))) # nonempty=True['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n']>>> import functools, operator>>> functools.reduce(operator.add, open('script2.py'))'import sys\nprint(sys.path)\nx = 2\nprint(x ** 32)\n'

essentially everything in Python’s built-in toolset that scans an object from left to right is defined to use the iteration protocol on the subject object.

>>> list(open('script2.py'))['import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n']>>> tuple(open('script2.py'))('import sys\n', 'print(sys.path)\n', 'x = 2\n', 'print(x ** 32)\n')>>> '&&'.join(open('script2.py'))'import sys\n&&print(sys.path)\n&&x = 2\n&&print(x ** 32)\n'

>>> def f(a, b, c, d): print(a, b, c, d, sep='&')...>>> f(1, 2, 3, 4)1&2&3&4>>> f(*[1, 2, 3, 4]) # Unpacks into arguments1&2&3&4

>>> X = (1, 2)>>> Y = (3, 4)>>>>>> list(zip(X, Y)) # Zip tuples: returns an iterable[(1, 3), (2, 4)>>> A, B = zip(*zip(X, Y)) # Unzip a zip!>>> A(1, 2)>>> B(3, 4)

3.New Iterables in Python 3.X

The range Iterable：

C:\code> c:\python33\python>>> R = range(10) # range returns an iterable, not a list>>> Rrange(0, 10)>>> I = iter(R) # Make an iterator from the range iterable>>> next(I) # Advance to next result0 # What happens in for loops, comprehensions, etc.>>> next(I)1>>> next(I)2>>> list(range(10)) # To force a list if required[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

range objects in 3.X support only iteration,indexing, and the len function.

>>> len(R) # range also does len and indexing, but no others10>>> R[0]0>>> R[-1]9>>> next(I) # Continue taking from iterator, where left off3>>> I.__next__() # .next() becomes .__next__(), but use new next()4

The map, zip, and filter Iterables：

>>> M = map(abs, (-1, 0, 1)) # map returns an iterable, not a list>>> M<map object at 0x00000000029B75C0>>>> next(M) # Use iterator manually: exhausts results1 # These do not support len() or indexing>>> next(M)0>>> next(M)1>>> next(M)StopIteration>>> for x in M: print(x) # map iterator is now empty: one pass only...>>> M = map(abs, (-1, 0, 1)) # Make a new iterable/iterator to scan again>>> for x in M: print(x) # Iteration contexts auto call next()...101>>> list(map(abs, (-1, 0, 1))) # Can force a real list if needed[1, 0, 1]

>>> Z = zip((1, 2, 3), (10, 20, 30)) # zip is the same: a one-pass iterator>>> Z<zip object at 0x0000000002951108>>>> list(Z)[(1, 10), (2, 20), (3, 30)]>>> for pair in Z: print(pair) # Exhausted after one pass...>>> Z = zip((1, 2, 3), (10, 20, 30))>>> for pair in Z: print(pair) # Iterator used automatically or manually...(1, 10)(2, 20)(3, 30)>>> Z = zip((1, 2, 3), (10, 20, 30)) # Manual iteration (iter() not needed)>>> next(Z)(1, 10)>>> next(Z)(2, 20)

>>> filter(bool, ['spam', '', 'ni'])<filter object at 0x00000000029B7B70>>>> list(filter(bool, ['spam', '', 'ni']))['spam', 'ni']

Multiple Versus Single Pass Iterators：

>>> R = range(3) # range allows multiple iterators>>> next(R)TypeError: range object is not an iterator>>> I1 = iter(R)>>> next(I1)0>>> next(I1)1>>> I2 = iter(R) # Two iterators on one range>>> next(I2)0>>> next(I1) # I1 is at a different spot than I22

multiple iterators are usually supported by returning new objects for the iter call;

a single iterator generally means an object returns itself

>>> Z = zip((1, 2, 3), (10, 11, 12))>>> I1 = iter(Z)>>> I2 = iter(Z) # Two iterators on one zip>>> next(I1)(1, 10)>>> next(I1)(2, 11)>>> next(I2) # (3.X) I2 is at same spot as I1!(3, 12)>>> M = map(abs, (-1, 0, 1)) # Ditto for map (and filter)>>> I1 = iter(M); I2 = iter(M)>>> print(next(I1), next(I1), next(I1))1 0 1>>> next(I2) # (3.X) Single scan is exhausted!StopIteration>>> R = range(3) # But range allows many iterators>>> I1, I2 = iter(R), iter(R)>>> [next(I1), next(I1), next(I1)][0 1 2]>>> next(I2) # Multiple active scans, like 2.X lists0

Dictionary View Iterables：

in Python 3.X the dictionary keys, values, and items methods return iterable view objects that generate result items one at a time

>>> D = dict(a=1, b=2, c=3)>>> D{'a': 1, 'b': 2, 'c': 3}>>> K = D.keys() # A view object in 3.X, not a list>>> Kdict_keys(['a', 'b', 'c'])>>> next(K) # Views are not iterators themselvesTypeError: dict_keys object is not an iterator>>> I = iter(K) # View iterables have an iterator,>>> next(I) # which can be used manually,'a' # but does not support len(), index>>> next(I)'b'>>> for k in D.keys(): print(k, end=' ') # All iteration contexts use auto...a b c

3.X dictionaries still are iterables themselves, with an iterator that returns successive keys：

>>> D # Dictionaries still produce an iterator{'a': 1, 'b': 2, 'c': 3} # Returns next key on each iteration>>> I = iter(D)>>> next(I)'a'>>> next(I)'b'>>> for key in D: print(key, end=' ') # Still no need to call keys() to iterate... # But keys is an iterable in 3.X too!a b c

4. Other Iteration Topics

• User-defined functions can be turned into iterable generator functions, with yield statements.
• List comprehensions morph into iterable generator expressions when coded in parentheses.
• User-defined classes are made iterable with __iter__ or __getitem__ operator overloading.