TopolearnCumulative sums, deltas between readings and processing records in batches are everyday stream operations.
Three iterators for sequences of related values:
accumulate(iterable, func=operator.add, initial=None) yields running results: accumulate([1, 2, 3]) gives 1, 3, 6. Pass another binary function for running maximum (max), product (operator.mul) or any fold where you want every intermediate state. initial prepends a starting value, so the output is one item longer.
pairwise(iterable) (3.10+) yields overlapping pairs of neighbours: pairwise("abcd") gives ab, bc, cd. It's ideal for differences between consecutive readings, checking whether a sequence is sorted, or building edges from a path. An input with fewer than two items yields nothing.
batched(iterable, n) (3.12+) yields tuples of up to n items: the last batch may be shorter. Python 3.13 adds strict=True, which raises ValueError if the final batch is incomplete. Use it to send records to an API or database in chunks of 500, or to process a big file in blocks. It reads lazily, so only one batch is in memory at a time.
Before 3.12, people used the "grouper" recipe with zip_longest, which pads the last batch with a fill value; batched is clearer.
Example
import operator
from itertools import accumulate, batched, pairwise
sales = [5, 3, 8, 2]
print(list(accumulate(sales))) # [5, 8, 16, 18]
print(list(accumulate(sales, max))) # [5, 5, 8, 8]
print(list(accumulate(sales, operator.mul, initial=1))) # [1, 5, 15, 120, 240]
temps = [20.5, 21.0, 19.5, 22.0]
print([round(b - a, 1) for a, b in pairwise(temps)]) # [0.5, -1.5, 2.5]
print(all(a <= b for a, b in pairwise([1, 2, 2, 5]))) # True
ids = range(1, 8)
for batch in batched(ids, 3):
print(batch)
# (1, 2, 3)
# (4, 5, 6)
# (7,)
try:
list(batched("abcde", 2, strict=True)) # 3.13+
except ValueError as e:
print(e) # batched(): incomplete batchRemember
accumulateyields running totals or any running foldpairwisegives overlapping neighbour pairs (3.10+)batched(it, n)chunks lazily;strict=True(3.13) rejects a short last batch
Watch out for
- Expecting batched to pad the last chunk; it simply yields a shorter tuple.