TopolearnThese bugs produce wrong results without errors, so experts learn to recognise them on sight.
Iterator bugs rarely crash; they quietly produce less data.
Silent exhaustion. A function that iterates its argument twice (if any(...) then for x in ...) works with lists and silently fails with iterators. Either document "needs a reusable iterable" (annotate as Collection or Sequence), or convert at the start with items = list(items). Checking iter(x) is x detects one-shot iterators.
StopIteration inside generators. Since PEP 479 (Python 3.7), a StopIteration escaping a generator body is converted to RuntimeError. A bare next(it) inside a generator on an empty iterator therefore crashes instead of ending the generator. Use next(it, default) or catch StopIteration and return.
Late binding. Generator expressions evaluate their first iterable immediately but everything else lazily. Variables referenced in the body are looked up when items are produced, so changing them before consumption changes the results.
Stale views. Lazy results built on mutable data reflect the data at consumption time. Mutating a list while an iterator walks it skips or repeats items; mutating a dict raises RuntimeError.
Partially consumed sources. zip may pull one extra item from an earlier argument before noticing a later one is exhausted; that item is lost.
Example
def total_and_max(items):
return sum(items), max(items, default=None)
print(total_and_max([3, 1, 4])) # (8, 4)
print(total_and_max(iter([3, 1, 4]))) # (8, None) -- exhausted by sum
def firsts(iterables):
for it in iterables:
yield next(iter(it)) # StopIteration on an empty one
try:
list(firsts([[1], [], [3]]))
except RuntimeError as e:
print(e) # generator raised StopIteration
factor = 2
doubled = (x * factor for x in [1, 2, 3])
factor = 10 # changed before consumption
print(list(doubled)) # [10, 20, 30]
a, b = iter([1, 2, 3]), iter(["x"])
print(list(zip(a, b)), list(a)) # [(1, 'x')] [3] -- 2 was lostRemember
- Functions that iterate twice need re-iterable inputs; convert or document
- StopIteration leaking from a generator becomes RuntimeError (PEP 479)
- Lazy expressions read variables and data at consumption time
Watch out for
- Calling bare
next()inside a generator without a default.