TopolearnComparisons like 'share of the group total' or 'rank within category' need group results next to each row.
expr.over("group") evaluates an expression within each group but returns a value for every row, like SQL's OVER (PARTITION BY ...). Rows are not collapsed, so you can use it in with_columns or select alongside ordinary columns.
- Broadcast an aggregate:
pl.col("sales").sum().over("region")puts the region total on each row. - Shares:
pl.col("sales") / pl.col("sales").sum().over("region"). - Rank within group:
pl.col("sales").rank("dense", descending=True).over("region"). - Running values:
pl.col("sales").cum_sum().over("region", order_by="day"). - Previous row:
pl.col("price").shift(1).over("ticker", order_by="ts").
Several partition keys work: .over("region", "year"). The order_by= argument sorts within each group just for that computation, so you don't have to pre-sort the whole frame.
mapping_strategy controls how results map back: the default "group_to_rows" puts results back on the original rows; "explode" is faster but reorders rows by group; "join" puts a list per group on each row.
Filtering by a window value is a classic pattern: keep the top two per group with df.filter(pl.col("sales").rank("ordinal", descending=True).over("region") <= 2).
Example
import polars as pl
df = pl.DataFrame({
"region": ["N", "N", "N", "S", "S"],
"day": [3, 1, 2, 1, 2],
"sales": [30, 10, 20, 5, 15],
})
out = df.with_columns(
share=(pl.col("sales") / pl.col("sales").sum().over("region")).round(2),
rank=pl.col("sales").rank("dense", descending=True).over("region"),
running=pl.col("sales").cum_sum().over("region", order_by="day"),
)
print(out.rows())
# [('N', 3, 30, 0.5, 1, 60), ('N', 1, 10, 0.17, 3, 10), ('N', 2, 20, 0.33, 2, 30),
# ('S', 1, 5, 0.25, 2, 5), ('S', 2, 15, 0.75, 1, 20)]Remember
.over(keys)= per-group computation, one value per row- Use
order_by=for running totals and lags within groups - Filter on a window rank for top-n per group
Watch out for
- Using group_by + join to broadcast totals, when one
.over()does it.