TopolearnMost real questions are about groups and about whether an effect differs between groups.
A categorical predictor with k levels becomes k - 1 dummy columns. The missing level is the reference, and each coefficient is the difference from it. By default the reference is the first level alphabetically, which may not be meaningful. Choose it with treatment coding: C(city, Treatment(reference="Rome")). Other contrasts are available, such as Sum (deviations from the grand mean).
An interaction lets the effect of one variable depend on another. price ~ area * C(city) fits a separate area slope per city: the area coefficient is the slope in the reference city, and area:C(city)[T.Oslo] is how much steeper or flatter it is in Oslo. With interactions, main effects alone are no longer "the effect of area"; they are conditional on the reference level (or on zero for numeric variables, which is why centring numeric variables helps).
To test whether a whole categorical factor or interaction matters, look at joint tests rather than individual p-values:
res.wald_test_terms()tests each term (all its dummies together),sm.stats.anova_lm(res_small, res_big)compares nested OLS models with an F-test,res.f_test("area:C(city)[T.Oslo] = 0")tests any linear hypothesis.
Too many interactions quickly overfit; add them because you have a reason to expect the effect to differ.
Example
import numpy as np
import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf
rng = np.random.default_rng(5)
df = pd.DataFrame({"area": rng.uniform(40, 150, 300),
"city": rng.choice(["Oslo", "Rome", "Lima"], 300)})
slope = df["city"].map({"Lima": 1.0, "Oslo": 3.0, "Rome": 2.0})
df["price"] = 40 + slope * df["area"] + rng.normal(0, 15, 300)
f = "price ~ area * C(city, Treatment(reference='Rome'))"
res = smf.ols(f, data=df).fit()
print(res.params.filter(like="area").round(2).tolist()) # [2.04, -1.08, 1.0]
# Rome slope 2.0; Lima is 1.0 lower; Oslo is 1.0 higher
small = smf.ols("price ~ area + C(city)", data=df).fit()
print(sm.stats.anova_lm(small, res).round(3)[["F", "Pr(>F)"]].iloc[1].tolist())
# [448.065, 0.0] (interaction clearly needed)Remember
- Each dummy coefficient is a difference from the reference level
- Set the reference with
C(x, Treatment(reference=...)) - Test whole factors and interactions jointly with F or Wald tests
Watch out for
- Interpreting a main effect as the overall effect when an interaction with it is in the model.