TopolearnConstraints encode domain knowledge, improve generalisation and make models acceptable to regulators.
Sometimes you know how a feature should act: risk should never decrease as debt increases; price should not fall as size grows. An unconstrained model may still learn small reversals from noise, which looks wrong to users and can generalise badly.
Monotonic constraints: monotone_constraints gives each feature 1 (non-decreasing), -1 (non-increasing) or 0 (unconstrained). With a DataFrame you can pass a dict: monotone_constraints={"debt": 1, "income": -1}. XGBoost then only accepts splits whose child values respect the order, and propagates bounds down the tree. The constraint holds for every prediction, holding the other features fixed.
Interaction constraints: interaction_constraints=[["age", "income"], ["region", "channel"]] lets features interact only within each group: a tree branch that split on age may later split only on age or income. That makes the model closer to an additive structure that is easier to explain, and can reduce overfitting.
Constraints usually cost a little training accuracy and can gain test accuracy when they reflect reality. Verify them with partial dependence plots, or by sweeping one feature over a grid on many rows and checking that predictions are monotone.
Both work with the hist tree method used by default, and with the native and scikit-learn APIs.
Example
import numpy as np
import pandas as pd
from xgboost import XGBRegressor
rng = np.random.default_rng(2)
n = 2000
X = pd.DataFrame({"size": rng.uniform(30, 200, n), "age": rng.uniform(0, 50, n)})
y = 2.0 * X["size"] - 0.5 * X["age"] + rng.normal(0, 40, n) # noisy
free = XGBRegressor(n_estimators=300, max_depth=5).fit(X, y)
mono = XGBRegressor(n_estimators=300, max_depth=5,
monotone_constraints={"size": 1, "age": -1}).fit(X, y)
grid = pd.DataFrame({"size": np.linspace(30, 200, 200), "age": 10.0})
for m in (free, mono):
pred = m.predict(grid)
print((np.diff(pred) < -1e-6).sum()) # number of decreases as size grows
# 76 (unconstrained model wiggles down and up)
# 0 (monotone in size)Remember
monotone_constraintswith 1/-1/0 per feature (dict for DataFrames)interaction_constraintslists groups of features allowed to interact- Verify constraints with a sweep or partial dependence
Watch out for
- Constraining a feature whose true relationship is not monotonic, which costs accuracy.