Cassandra Hot Partitions

A partition lives on RF nodes. If one partition receives a disproportionate share of traffic, those nodes saturate while the rest of the cluster is idle — and adding nodes does not help.

There are two distinct problems: partitions that are too busy, and partitions that are too large.

Finding them

# Largest partitions per table, with percentiles.
nodetool tablehistograms shop events_by_user
 
# Max partition size and cell counts.
nodetool tablestats shop.events_by_user

Partition Size (bytes)
  50%: 1109
  95%: 14237
  99%: 62911
  Max: 1258291200        <-- 1.2 GB in one partition

A maximum far above the 99th percentile means a small number of pathological partitions — usually one tenant, one device or one popular entity.

Uneven Load in nodetool status while Owns is even is the cluster-level symptom of the same problem.

Spreading a busy partition

Add a bucket to the partition key. Time is the usual bucket:

-- Before: unbounded and hot for active users.
PRIMARY KEY (user_id, event_time)
 
-- After: one partition per user per day.
PRIMARY KEY ((user_id, day), event_time)

The application queries the buckets it needs. Choose granularity so a partition holds a bounded number of rows at your peak write rate — hourly for very high volume, monthly for low.

Add a synthetic shard for a single hot key. When one entity is genuinely hot:

PRIMARY KEY ((tenant_id, shard), event_time)
-- shard = hash(event_id) % 16 on write; read all 16 shards and merge.

Reads become 16 queries instead of one. That is the trade: fan-out on read in exchange for spreading the write load.

Reconsider the key entirely. A partition key with low cardinality — status, country, boolean flag — cannot spread across a cluster at all. It is not a tuning problem; the key is wrong.

Bounding partition size

Detecting at the application layer

Cassandra does not tell you which partitions are queried most. Instrument the client:

// Record the partition key with each query so hot keys are visible in metrics.
meter.counter("cassandra.query", "table", "events_by_user", "partition", userId).increment();

Sampling query traces also works:

TRACING ON;
SELECT * FROM events_by_user WHERE user_id = ? AND bucket = ?;
TRACING OFF;

Tracing shows the coordinator, the replicas contacted and the time spent at each — useful for a specific slow query, and too expensive to leave on.