Cardinality

A series is a unique combination of metric name and label values. Cardinality is the number of distinct series.

http_requests_total{method="GET", status="200", endpoint="/api/orders"}   → one series
http_requests_total{method="POST", status="500", endpoint="/api/orders"}  → another

Points per series are cheap: they compress well and append sequentially. Series are expensive: each needs index entries, in-memory state and metadata. Cardinality, not point count, is the binding constraint.

Cardinality multiplies

Total series is the product of label value counts, per metric:

methods (5) × statuses (8) × endpoints (200) × instances (50) = 400,000 series

Adding one label with 1,000 values makes it 400,000,000. This is why cardinality problems appear suddenly: a single deploy adds a label and multiplies the whole space.

Measuring it

VictoriaMetrics exposes cardinality analysis directly, and its UI includes a cardinality explorer:

/api/v1/status/tsdb

Prometheus-compatible query for the top metrics by series count:

topk(10, count by (__name__)({__name__=~".+"}))

InfluxDB:

SHOW SERIES CARDINALITY;
SHOW TAG KEY CARDINALITY;

TimescaleDB — cardinality is table cardinality; count distinct combinations of the identifying columns:

SELECT count(*) FROM (SELECT DISTINCT device_id, metric FROM readings) s;

Controlling it

Drop labels at ingestion. Filter high-cardinality labels before they reach storage rather than after:

# Prometheus relabeling: remove a label entirely.
metric_relabel_configs:
  - regex: 'request_id|trace_id|session_id'
    action: labeldrop

Aggregate before storing. If per-instance detail is not queried, aggregate to the service level at ingest. Recording rules and stream aggregation both do this.

Bucket continuous values. Store a latency bucket, not a latency value, as a label.

Set limits so the failure is bounded. Most engines can reject new series past a threshold, which turns a slow death into a visible error:

-maxLabelsPerTimeseries=30
-storage.maxHourlySeries=1000000
-storage.maxDailySeries=5000000

Churn

Series churn — series that appear and disappear rather than persisting — is a distinct problem. Every deploy that changes a version or pod label creates a full set of new series while the old ones remain queryable for the retention period. Steady-state cardinality may look fine while total indexed series grows every day.