TopolearnAutomatic alignment prevents mismatched-row bugs but creates surprise NaNs if you don't expect it.
The index is not just decoration. When you combine two Series or DataFrames, pandas aligns them on their labels first, then operates on matching pairs. Labels present in only one operand produce NaN in the result.
a = pd.Series([1, 2], index=["x", "y"])
b = pd.Series([10, 20], index=["y", "z"])
a + b # x NaN, y 12, z NaNThe same happens when assigning a Series as a column: df["new"] = other_series matches rows by index label, not by position. If the indexes don't line up you get NaNs, or silently wrong rows if they happen to overlap in the wrong way.
This is powerful when it's what you want: subtracting last year's sales from this year's by product just works, whatever the row order. It's surprising when you think positionally, for example after filtering or sorting one side only.
Tools to stay in control:
a.add(b, fill_value=0)(alsosub,mul,div) treats missing labels as 0 instead of producing NaN.a.align(b, join="inner")returns both, restricted to common labels.reset_index(drop=True)throws away the labels and renumbers 0..n-1, when you really mean "by position"..to_numpy()on the right-hand side assigns by position, if the lengths match.df.reindex(new_labels)conforms to a given set of labels, inserting NaN for new ones.
Indexes can contain duplicate labels, which makes alignment produce every matching combination. Check df.index.is_unique when results suddenly have more rows than expected.
Example
import pandas as pd
this_year = pd.Series({"tea": 120, "coffee": 300, "cocoa": 40})
last_year = pd.Series({"coffee": 250, "tea": 100, "juice": 90})
print(this_year - last_year)
# cocoa NaN
# coffee 50.0
# juice NaN
# tea 20.0
print(this_year.sub(last_year, fill_value=0)) # cocoa 40, juice -90, ...
df = pd.DataFrame({"v": [1, 2, 3]}, index=[10, 20, 30])
s = pd.Series([100, 200, 300]) # index 0, 1, 2
df["by_label"] = s # no matching labels!
df["by_position"] = s.to_numpy()
print(df)
# v by_label by_position
# 10 1 NaN 100
# 20 2 NaN 200
# 30 3 NaN 300Remember
- Binary operations align on index labels before computing
- Labels missing on one side give NaN; use
fill_value=to avoid that - Assigning a Series as a column aligns by label, not position
- Use
reset_index(drop=True)or.to_numpy()when you mean position
Watch out for
- Assigning a filtered or re-sorted Series to a column of another DataFrame and getting NaNs or misplaced rows.