Aggregations and the axis argument

NumPy · Essentials · 12 min

Summaries per row, per column or over everything are the most common analysis step, and axis mistakes are the most common bug.

Aggregations collapse many values into fewer: sum, mean, median, min, max, std, var, prod, argmin/argmax (position of the extreme), any/all, and the cumulative versions cumsum/cumprod. Most exist as both functions (np.sum(a)) and methods (a.sum()).

Without arguments they reduce over every element. The axis argument chooses which axis to collapse:

  • For a 2-D table of shape (rows, cols), axis=0 collapses the rows and gives one value per column, shape (cols,).
  • axis=1 collapses the columns and gives one value per row, shape (rows,).

A reliable way to remember: the axis you name is the one that disappears from the shape. With a shape (10, 24, 7) array, a.mean(axis=1) has shape (10, 7). You can pass a tuple, axis=(1, 2), to collapse several at once.

keepdims=True keeps the collapsed axis with length 1, so the result broadcasts back against the original. That is how you normalise each row: a / a.sum(axis=1, keepdims=True).

Two details trip people up. np.std uses the population formula by default (ddof=0); pass ddof=1 for the sample standard deviation that pandas and most statistics texts use. And argmax without an axis returns an index into the flattened array; np.unravel_index converts it back to a (row, col) pair.

If the data contains nan, regular aggregations return nan; the np.nansum, np.nanmean family skips missing values.

Example

import numpy as np

sales = np.array([[3, 7, 2],     # rows: stores, cols: days
                  [5, 1, 8]])
print(sales.sum())               # 26
print(sales.sum(axis=0))         # [ 8  8 10]  per day (column)
print(sales.sum(axis=1))         # [12 14]     per store (row)
print(sales.max(axis=1))         # [7 8]

i = sales.argmax()               # index in the flattened array
print(i, np.unravel_index(i, sales.shape))   # 5 (np.int64(1), np.int64(2))

share = sales / sales.sum(axis=1, keepdims=True)
print(share.round(2))            # each row sums to 1

x = np.array([2.0, 4.0, 4.0, 6.0])
print(x.std(), x.std(ddof=1))    # 1.414... 1.632...
print(np.nanmean([1.0, np.nan, 3.0]))   # 2.0

Remember

  • axis=k collapses axis k: it vanishes from the result's shape
  • On a 2-D table, axis=0 gives per-column results, axis=1 per-row
  • keepdims=True keeps a length-1 axis so results broadcast back
  • std defaults to ddof=0; use nan* functions to skip missing values

Watch out for

  • Mixing up axis=0 and axis=1: check the result's shape against what you expect.

Check yourself

  1. `a` has shape (5, 3, 4). What is the shape of `a.sum(axis=(0, 2))`?
  2. For a (100, 4) array of 100 samples with 4 features, which gives the mean of each feature?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: numpy.org/doc/stable/reference/generated/numpy.sum.html