TopolearnThe most valuable metrics are the ones only your application knows: business events, queue depth, request latency.
Official client libraries exist for Go, Java, Python, Ruby and Rust, and community ones for most other languages. The pattern is the same everywhere:
- Create metrics once, at module level, with a name, help text and the label names.
- Update them in code:
inc()counters,set()/inc()/dec()gauges,observe()histograms (or use a timer helper). - Expose
/metrics: a built-in HTTP server, or a handler mounted in your web framework.
What to instrument, following the RED method for services: Rate of requests, Errors, Duration (a histogram). For resources such as queues and pools, the USE method: utilization, saturation, errors.
Label hygiene matters most here: use the route template (/users/{id}), never the actual path, and keep status codes or status classes. Choose histogram buckets around your latency objectives (for example 50 ms, 100 ms, 250 ms, 500 ms, 1 s).
Multi-process servers such as Gunicorn need the client library's multiprocess mode, since each worker has its own memory. For batch jobs that finish before a scrape, push final values to the Pushgateway instead.
In Python, prometheus_client also exposes default process and Python runtime metrics automatically.
Example
# pip install prometheus-client
import random, time
from prometheus_client import Counter, Histogram, start_http_server
REQUESTS = Counter("shop_requests_total", "Requests handled", ["route", "status"])
LATENCY = Histogram("shop_request_duration_seconds", "Request latency",
["route"], buckets=[0.05, 0.1, 0.25, 0.5, 1])
def handle(route):
with LATENCY.labels(route).time(): # observes the duration
time.sleep(random.random() / 10)
REQUESTS.labels(route, "200").inc()
start_http_server(8000) # serves /metrics on :8000
for _ in range(50):
handle("/products/{id}") # route template, not real path
# curl -s localhost:8000/metrics | grep shop_requests
# shop_requests_total{route="/products/{id}",status="200"} 50.0Remember
- Define metrics once, update them in code, expose /metrics
- RED: rate, errors, duration histogram for every service
- Label with route templates and bounded values
Watch out for
- Creating a new metric object per request, causing duplicate registration errors or leaks.