TopolearnA point prediction without an interval hides how wrong it could be.
res.predict(new_data) returns point predictions. For uncertainty use res.get_prediction(new_data) and .summary_frame(alpha=0.05), which returns a DataFrame with:
mean: the predicted value,mean_ci_lower,mean_ci_upper: the confidence interval for the average response at those predictor values,obs_ci_lower,obs_ci_upper: the prediction interval for a single new observation.
The difference matters. The confidence interval only reflects uncertainty in the estimated coefficients, and shrinks as you collect more data. The prediction interval adds the irreducible noise of individual observations, so it is always wider and does not shrink below the noise level. "What is the average price of 100 m² flats?" needs the first; "what will this flat sell for?" needs the second.
With a formula model, pass a DataFrame with the original column names and the formula is applied automatically. With the array API, new_data must have the same columns, including the constant: sm.add_constant(new_X, has_constant="add") is needed when new_X has a single row, because add_constant skips columns it thinks are already constant.
Intervals assume the model is correct and residuals are roughly normal with constant variance. Far outside the range of the training data (extrapolation) they are optimistic.
Example
import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
rng = np.random.default_rng(3)
df = pd.DataFrame({"area": rng.uniform(40, 150, 200)})
df["price"] = 30 + 2.5 * df["area"] + rng.normal(0, 20, 200)
res = smf.ols("price ~ area", data=df).fit()
new = pd.DataFrame({"area": [60, 100, 140]})
frame = res.get_prediction(new).summary_frame(alpha=0.05)
print(frame[["mean", "mean_ci_lower", "mean_ci_upper", "obs_ci_lower", "obs_ci_upper"]].round(1))
# mean mean_ci_lower mean_ci_upper obs_ci_lower obs_ci_upper
# 0 181.5 177.2 185.9 142.4 220.6
# 1 280.5 277.7 283.2 241.5 319.4
# 2 379.4 374.5 384.3 340.2 418.6Remember
get_prediction(...).summary_frame()gives means and both interval types- Confidence intervals cover the average response; prediction intervals cover one new observation
- Prediction intervals are always wider
Watch out for
- Quoting the confidence interval as the range for an individual future value.