TopolearnLinear regression is the baseline for every regression task and the simplest model you can explain.
A regressor predicts a continuous value. LinearRegression fits y ≈ w1*x1 + w2*x2 + ... + b by ordinary least squares, minimising the sum of squared errors. After fit the weights are in coef_ (one per feature) and the intercept in intercept_.
Its score(X, y) returns R², the fraction of the target's variance the model explains: 1.0 is perfect, 0.0 means no better than always predicting the mean, and it can be negative on test data for a bad model.
Reading coefficients needs care:
- A coefficient is the change in prediction for a one-unit increase in that feature with the others held fixed, so its size depends on the feature's unit. Compare importance only after standardising features.
- Strongly correlated features make individual coefficients unstable, even when predictions are fine.
- Linear means linear in the parameters. You can still model curves by adding transformed features such as squares or logs.
LinearRegression has no regularisation and no hyperparameters to tune, which makes it an honest baseline. When you have many features or correlated features, the regularised versions Ridge and Lasso usually generalise better. For a target that spans orders of magnitude (prices, counts), fitting on log(y) often helps.
Example
from sklearn.datasets import load_diabetes
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=0)
reg = LinearRegression().fit(X_tr, y_tr)
print(round(reg.intercept_, 1)) # 153.1
print(dict(zip(X.columns[:3], reg.coef_[:3].round(1).tolist())))
# {'age': -43.3, 'sex': -208.7, 'bmi': 593.4}
print(round(reg.score(X_tr, y_tr), 3)) # 0.555 (train R2)
print(round(reg.score(X_te, y_te), 3)) # 0.359 (test R2)
print(reg.predict(X_te[:2]).round(0)) # [242. 250.]Remember
coef_andintercept_define the fitted linescorereturns R², which can be negative on test data- Compare coefficient sizes only on standardised features
Watch out for
- Reading a large coefficient as an important feature when the feature simply has small units.