Skip to content
Navigation

Type at least two characters. Search covers page titles, headings, tags and database names.

↑ ↓ to navigateEnter to openEsc to close0 pages
Monitoringintermediate

Prometheus Monitoring

Scraping database metrics with exporters, the permissions they need, and keeping cardinality under control.

2 min readIntermediateUpdated Edit this page

Prometheus scrapes metrics endpoints on an interval and stores them as time series. For databases, the endpoint is usually an exporter that translates the engine's own statistics.

Exporters

EngineExporterNotes
PostgreSQLpostgres_exporterSupports custom queries; grant pg_monitor
MySQL / MariaDBmysqld_exporterVerify collectors against MariaDB
Redis / Valkeyredis_exporterCan also export per-key metrics — use sparingly
MongoDBmongodb_exporterNeeds clusterMonitor
CassandraJMX exporter agentRuns in-process as a Java agent
ScyllaDBNative /metricsNo JMX; metric names differ from Cassandra
ClickHouseNative <prometheus> endpointPlus custom queries for table-level facts
Elasticsearchelasticsearch_exporterOr the native monitoring integration
# prometheus.yml
scrape_configs:
  - job_name: postgres
    scrape_interval: 15s
    static_configs:
      - targets: ['db-primary:9187', 'db-replica-a:9187']
        labels: { cluster: shop, env: prod }

Least privilege for the exporter

-- PostgreSQL: pg_monitor grants read access to statistics without data access.
CREATE ROLE metrics LOGIN PASSWORD :'password';
GRANT pg_monitor TO metrics;
-- MySQL
CREATE USER 'metrics'@'localhost' IDENTIFIED BY '…';
GRANT PROCESS, REPLICATION CLIENT, SELECT ON performance_schema.* TO 'metrics'@'localhost';

Custom queries

Exporters cover server-level counters. Table-level facts — bloat, per-table dead tuples, replication slot retention — usually need custom queries:

# postgres_exporter queries.yaml
pg_replication_slots:
  query: |
    SELECT slot_name, active::int AS active,
           pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS retained_bytes
    FROM pg_replication_slots
  metrics:
    - slot_name: { usage: "LABEL" }
    - active:    { usage: "GAUGE", description: "Whether the slot has a connected consumer" }
    - retained_bytes: { usage: "GAUGE", description: "WAL retained for this slot" }

Cardinality

Prometheus is a time-series database, and everything in Cardinality applies. The common database-monitoring mistakes:

  • Per-query-hash metrics. A label per normalised query multiplies series and grows unbounded.
  • Per-table metrics on a schema with thousands of tables.
  • Per-key metrics from redis_exporter.
  • Container ids or pod names as labels on a frequently redeployed service.

Drop what you do not query:

metric_relabel_configs:
  - source_labels: [__name__]
    regex: 'pg_stat_statements_.*'
    action: drop

Scrape interval

Fifteen seconds is a reasonable default. Shorter intervals multiply storage and load on the exporter; longer ones miss short spikes. Match the interval to the alert you intend to write — an alert with for: 2m gains nothing from five-second scrapes.