Learn Prometheus

Prometheus scrapes metrics from targets over HTTP, stores them as time series and evaluates queries and alerts on them.

A time series is a metric name plus a set of labels; each holds timestamped float samples. Learn naming and why labels matter.

The four metric types, what each is for, and why you almost never look at a counter's raw value.

Name metrics with a domain prefix, base units and the right suffix (_total, _seconds, _bytes, _info) so queries stay predictable.

Why a counter's absolute value is meaningless, how resets look, when a gauge is the right choice, and which functions fit each.

Start Prometheus, write prometheus.yml with scrape_configs, and check target health with the up metric.

Tune scrape_interval and scrape_timeout, understand honor_labels and honor_timestamps, limits, scrape protocols and how stale series disappear.

Monitor hosts, endpoints and third-party software with exporters such as node_exporter and blackbox_exporter.

Get CPU, memory, disk and network for hosts from node_exporter and for containers from cAdvisor, and write the standard queries for each.

Push final metrics from short-lived jobs to the Pushgateway, scrape it with honor_labels, and understand why it is not a general push endpoint.

Select series with label matchers, tell instant vectors from range vectors, and use offset and comparisons.

Combine matchers, match on __name__, test for missing labels, use anchored regexes safely and the 3.x quoted-name syntax.

Explore queries in the built-in UI, then call the same engine over HTTP with /api/v1/query and query_range.

Turn ever-growing counters into per-second rates and increases, handle counter resets, and pick a sensible range.

Aggregate with sum, avg, max, count and topk by labels, and combine vectors with on, ignoring and group_left.

Shift a selector back in time with offset, pin it to a fixed moment with @, and combine both for week-over-week comparisons.

The functions you reach for daily beyond rate: absent, changes and resets, delta, deriv and predict_linear, label_replace, clamp and the *_over_time family.

Turn any instant-vector expression into a range vector with [range:resolution] so you can apply *_over_time functions to computed values.

Compute percentiles from histogram buckets, aggregate them correctly, and understand their error and the native histogram alternative.

Add counters, gauges and histograms to an application with a client library and expose a /metrics endpoint.

Use client_golang with promauto and promhttp to expose /metrics, instrument HTTP handlers and register custom collectors.

Enable native histograms, expose them from client libraries over protobuf, and query them with histogram_quantile, histogram_fraction and friends.

Serve the UI and API over TLS with basic auth via the web config file, scrape targets securely, and understand that Prometheus has no authorization layer.

How the TSDB stores samples in memory and on disk, what the WAL protects, how compaction and retention work, and how to size it.

Find and fix cardinality explosions, run HA pairs, and scale beyond one server with sharding, federation, agent mode and remote write to Thanos or Mimir.

What changed in 3.x: UTF-8 metric names, the OTLP receiver, left-open range selectors, strict scrape protocols and the upgrade path from 2.x.

Precompute expensive or frequently used queries into new series with recording rules, and name them by convention.

Write alerting rules with for, labels and templated annotations, and design alerts on symptoms rather than causes.

Write annotations with Go templates, humanize values, link runbooks and dashboards, and shape Alertmanager notification templates.

Route alerts to the right receivers, group and deduplicate them, inhibit noise, and silence alerts during maintenance.

Feed synthetic series into recording and alerting rules with promtool test rules, and assert on outputs, labels and annotations in CI.

Define an SLO and error budget in PromQL, then page on burn rate over paired long and short windows instead of raw thresholds.

Discover targets dynamically from Kubernetes, clouds or files, and rewrite target and metric labels with relabel_configs.

Install the Prometheus Operator stack with Helm and declare scrape targets as ServiceMonitor, PodMonitor and PrometheusRule objects.