TopolearnSearch results, feeds and recommendations are judged by order, not by individual scores.
In learning to rank you have queries (a search, a user session), each with several candidate items labelled by relevance (for example 0-4 or clicked/not). The goal is to order items within each query, so only relative scores inside a query matter.
XGBRanker handles this with:
objective="rank:ndcg"(default),"rank:pairwise"or"rank:map",- a query id per row passed as
qid=tofit. Rows must be sorted by qid so each query's rows are contiguous, eval_metricsuch as"ndcg@10"or"map@5", evaluated per query and averaged.
Since XGBoost 2.0, pairwise objectives build pairs within a query according to lambdarank_pair_method ("topk" focuses on the top lambdarank_num_pair_per_sample positions, or "mean" samples pairs), and lambdarank_unbiased=True corrects for position bias in click data.
predict returns scores; sort items by score within each query to produce rankings. Features should describe the query-item pair (text match score, item popularity, user-item affinity).
Evaluate with ranking metrics per query, splitting train and test by query (GroupKFold or GroupShuffleSplit with qid as groups) so no query appears in both.
Example
import numpy as np
from sklearn.metrics import ndcg_score
from xgboost import XGBRanker
rng = np.random.default_rng(3)
n_queries, per_q = 200, 10
qid = np.repeat(np.arange(n_queries), per_q) # sorted query ids
X = rng.normal(size=(n_queries * per_q, 5))
relevance = np.clip(np.round(X[:, 0] + 0.5 * X[:, 1] + rng.normal(0, 0.5, len(X)) + 1), 0, 4)
train = qid < 150
ranker = XGBRanker(objective="rank:ndcg", n_estimators=200, learning_rate=0.1,
eval_metric="ndcg@5")
ranker.fit(X[train], relevance[train], qid=qid[train])
scores = ranker.predict(X[~train]).reshape(-1, per_q) # 50 test queries x 10 items
rel = relevance[~train].reshape(-1, per_q)
print(round(ndcg_score(rel, scores, k=5), 3)) # 0.946
print(round(ndcg_score(rel, rng.normal(size=rel.shape), k=5), 3)) # 0.538 (random order)Remember
- Pass
qidper row and keep rows sorted by query - Use
rank:ndcgwithndcg@kevaluation - Split train and test by query, never by row
Watch out for
- Passing rows in random order, so a query's items are not contiguous.