PromQL basics: selectors and vectors

Prometheus · Essentials · 12 min

PromQL is how you get anything out of Prometheus: graphs, dashboards and alerts are all PromQL expressions.

A PromQL expression evaluates to one of four types; the two you use constantly are:

  • Instant vector: one sample per series at a single point in time. node_memory_MemAvailable_bytes returns the latest value for every matching series.
  • Range vector: a window of samples per series, written with a duration in brackets: http_requests_total[5m]. Range vectors can't be graphed directly; they are passed to functions like rate().

Label matchers filter series inside braces:

  • = equal, != not equal,
  • =~ regex match, !~ regex non-match. Regexes are fully anchored: status=~"5.." matches 500 to 599, and "5" alone would only match exactly 5.

Examples: http_requests_total{job="api", method!="GET"} or {__name__=~"node_cpu.*"}.

Other basics:

  • offset 1h looks back in time: http_requests_total offset 1d.
  • Arithmetic and comparison operators work on vectors and scalars: node_filesystem_avail_bytes / node_filesystem_size_bytes < 0.1 keeps only filesystems with less than 10 percent free. Comparison operators filter series rather than returning true or false (add bool to get 0 or 1).
  • Durations use units such as 30s, 5m, 1h, 1d.

Example

# All series of the metric, latest value each (instant vector)
up
# Filter by labels; regex is fully anchored
http_requests_total{job="api", status=~"5..", method!="OPTIONS"}
# Range vector: the last 5 minutes of samples per series (input for functions)
http_requests_total{job="api"}[5m]
# Compare with the same time yesterday
node_load1 - node_load1 offset 1d
# Filesystems with less than 10% space left (filters series)
node_filesystem_avail_bytes{fstype!="tmpfs"} / node_filesystem_size_bytes{fstype!="tmpfs"} < 0.10
# Targets that are down
up == 0

Remember

  • Instant vector: latest sample per series; range vector: [5m] window per series
  • Matchers =, !=, =~, !~; regexes are fully anchored
  • Comparisons filter series unless you add bool

Watch out for

  • Writing status=~"5" and expecting all 5xx codes; use "5..".

Check yourself

  1. What does http_requests_total[5m] return?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: prometheus.io/docs/prometheus/latest/querying/basics/