Redis Pipelining

Redis commands are usually fast; the network round trip is not. Pipelining sends many commands without waiting for each reply, so N commands cost one round trip instead of N.

# Without pipelining: 1000 round trips.
for user_id in user_ids:
    r.hget(f"shop:user:{user_id}", "email")
 
# With pipelining: one round trip for the batch.
pipe = r.pipeline(transaction=False)
for user_id in user_ids:
    pipe.hget(f"shop:user:{user_id}", "email")
results = pipe.execute()

On a 0.5 ms network, a thousand sequential commands take about half a second of pure waiting. Pipelined, the same work is dominated by execution time — typically a small fraction of that.

Pipelining is not a transaction

Pipelined commands are not atomic. Other clients' commands can interleave with them, and there is no rollback if one fails. If you need atomicity, use transactions or a Lua script.

Keep batches bounded

Alternatives worth knowing

Variadic commands are better than pipelining the single-key form, because they are one command:

MGET key1 key2 key3
HMGET user:1001 name email
SADD tags:1001 a b c
DEL k1 k2 k3

Client-side batching in cluster mode must group keys by slot. Cluster clients generally split a pipeline per node automatically, but a multi-key command still requires all keys in one slot — see Redis Cluster.

Measuring the benefit

# Compare pipelined and non-pipelined throughput on your own network.
redis-benchmark -h redis.internal -t get,set -n 100000
redis-benchmark -h redis.internal -t get,set -n 100000 -P 16

The -P flag sets the pipeline depth. The difference between the two runs is the round-trip cost you are paying without it, and it tells you the depth beyond which you stop gaining.