Golden Signals for Databases
Latency, traffic, errors and saturation applied to databases, and what each one looks like per engine.
The four golden signals apply to databases with one adjustment: saturation matters more than anywhere else, because databases have hard ceilings that cause abrupt failure rather than gradual slowdown.
Latency
Query duration, as percentiles. p50, p95, p99 — never the average.
Measure it from two places. Server-side latency tells you what the database did; client-side latency includes connection acquisition, network and driver overhead. A 3 ms server-side query that takes 400 ms client-side is a pool problem, and only the client measurement reveals it.
Split latency by operation type. Reads and writes have different profiles and different causes when they degrade.
Traffic
Queries or transactions per second, split by type. Its value is mostly as context: a latency increase with rising traffic is capacity; a latency increase with flat traffic is a regression — a plan change, a lock, a failing disk.
Errors
- Connection failures and refusals.
- Query errors, by class: syntax, timeout, deadlock, serialization failure, permission.
- Replication errors and stopped replicas.
- Backup failures.
Distinguish the classes. A rising deadlock rate and a rising connection-refused rate are unrelated problems.
Saturation
The signal that predicts outages, because these resources fail abruptly at 100%:
Database-specific fifth signal: replication
Databases have a signal the original four do not cover: whether the copies of your data are current.
- Replication lag per replica, in bytes and in time.
- Replica state — streaming, stopped, recovering.
- Time since the last successful backup, and since the last verified restore.
These are silent failures. Nothing degrades when replication stops; you find out during a failover, which is the worst possible moment.
A single overview
One dashboard should answer "is this database healthy" without scrolling:
- Query latency percentiles, read and write.
- Traffic, by operation type.
- Error rate, by class.
- Saturation of every bounded resource, as a percentage with a trend.
- Replication lag and backup freshness.
Everything else belongs on a second screen, reached when the first one shows something is wrong.