Relevance scoring and tuning

Elasticsearch · Practical · 13 min

Users judge search by whether the best result is first, which takes deliberate relevance tuning.

Elasticsearch scores full-text matches with BM25 by default. For each query term in each field, the score grows with:

  • term frequency: how often the term occurs in the field (with diminishing returns, controlled by k1),
  • inverse document frequency: rarer terms across the index count more,
  • shorter fields: a match in a short title beats one in a long body (length normalization, controlled by b).

Scores are relative, only comparable within one query, so never apply fixed score thresholds across queries.

Tuning tools, roughly in the order to try them:

  • field boosts in multi_match ("title^3"), and type: "best_fields", "most_fields" or "cross_fields" depending on how fields relate,
  • phrase proximity: add a match_phrase with slop in should to reward words appearing together,
  • business signals with function_score or script_score: blend popularity, recency (gauss decay on a date) or rating into the score; rank_feature fields are an efficient alternative,
  • synonyms at search time.

Use "explain": true to see each document's score breakdown, and evaluate changes against a set of real queries with judged results (the _rank_eval API) rather than eyeballing one query.

For hybrid lexical and semantic search, scores from different methods aren't comparable; combine result lists with reciprocal rank fusion (RRF) instead.

Example

GET /articles/_search
{
  "query": {
    "function_score": {
      "query": { "bool": {
        "must":   [ { "multi_match": { "query": "solar battery storage",
                                       "fields": ["title^3", "body"], "type": "best_fields" } } ],
        "should": [ { "match_phrase": { "title": { "query": "battery storage", "slop": 1 } } } ]
      } },
      "functions": [
        { "gauss": { "published": { "origin": "now", "scale": "30d", "decay": 0.5 } } },
        { "field_value_factor": { "field": "likes", "modifier": "log1p", "missing": 0 } }
      ],
      "score_mode": "sum",
      "boost_mode": "multiply"
    }
  },
  "explain": false
}

Remember

  • BM25: term frequency, rarity (IDF) and field length drive scores
  • Scores are relative to a query; don't threshold them
  • Boost fields, reward phrases, then blend business signals
  • Measure changes with judged queries, not single examples

Watch out for

  • Tuning boosts based on one query and making ten others worse.

Check yourself

  1. All else equal, which document scores higher for the term "zephyr" under BM25?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-similarity.html