TopolearnPCA speeds up models, removes redundancy and makes high-dimensional data plottable.
Principal component analysis finds new axes (components) that are linear combinations of the original features, ordered so the first captures the most variance, the second the most of what remains, and so on, all mutually uncorrelated. Keeping the first few components gives a compact representation.
PCA(n_components=2)keeps two components, good for plotting.PCA(n_components=0.95)keeps as many as needed to retain 95% of the variance.explained_variance_ratio_shows each component's share; its cumulative sum guides the choice.components_holds the loadings, andinverse_transformmaps back to the original space (approximately).
Scale first: PCA follows variance, so an unscaled feature in large units would dominate the first component. A make_pipeline(StandardScaler(), PCA(...), model) keeps everything fitted on training data.
PCA is linear and unsupervised: it keeps directions of high variance, which are not necessarily the ones that predict y. It helps most when features are many and correlated (pixels, sensor readings). For tree ensembles it often hurts, because it mixes features the trees could have split on individually. For visualising clusters, non-linear methods such as TSNE show more structure, but their output should not be used as model features.
For sparse text data use TruncatedSVD, which works without centring.
Example
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
X, y = load_digits(return_X_y=True) # 1797 images, 64 pixel features
Xs = StandardScaler().fit_transform(X)
pca = PCA(n_components=0.90).fit(Xs)
print(pca.n_components_) # 31
print(pca.explained_variance_ratio_[:3].round(3)) # [0.12 0.096 0.084]
X2 = PCA(n_components=2).fit_transform(Xs)
print(X2.shape) # (1797, 2), ready to scatter-plot
print(np.cumsum(pca.explained_variance_ratio_)[-1].round(3)) # 0.9Remember
- Components are ordered by explained variance and uncorrelated
n_components=0.95keeps enough components for 95% of variance- Scale before PCA and fit it inside a pipeline
Watch out for
- Assuming the top-variance components are the most predictive ones.