Skip to content
Navigation

Type at least two characters. Search covers page titles, headings, tags and database names.

↑ ↓ to navigateEnter to openEsc to close0 pages
PostgreSQLintermediate

PostgreSQL Memory Settings

How shared_buffers, work_mem, maintenance_work_mem and the OS page cache interact, and how to avoid sizing them into an out-of-memory kill.

3 min readIntermediateUpdated Edit this page

PostgreSQL memory falls into two classes: shared memory allocated once at startup, and per-backend memory allocated on demand. Only the second class can grow without bound.

Shared memory

shared_buffers is allocated at startup and never grows. It caches data and index pages; dirty pages are written by the checkpointer and background writer.

PostgreSQL relies on the OS page cache as a second tier, so allocating most of RAM to shared_buffers is counterproductive — the same pages end up cached twice and less memory remains for everything else. About a quarter of RAM is the conventional starting point.

Verify empirically rather than by rule:

SELECT sum(heap_blks_hit) * 100.0 / nullif(sum(heap_blks_hit + heap_blks_read), 0)
       AS cache_hit_percent
FROM pg_statio_user_tables;

A hit ratio consistently below roughly 95–99% on an OLTP workload suggests the working set no longer fits. Whether the fix is more shared_buffers or more RAM depends on the OS cache behaviour — check both before changing either.

Per-backend memory

This is where out-of-memory kills come from.

work_mem is the limit per sort, hash or materialise node, not per query. One query can contain many such nodes, and each parallel worker gets its own allowance. A rough worst case:

peak ≈ work_mem × nodes_per_query × (1 + parallel_workers) × concurrent_queries

With work_mem = 64MB, a query with three hash joins, four parallel workers and fifty concurrent executions, the arithmetic reaches hundreds of gigabytes. In practice not every node reaches its limit — but the mechanism is why a global increase to work_mem is a common cause of OOM.

Spilling to disk when work_mem is exceeded is not a failure — it is the safety valve. Enable log_temp_files = 0 to see which queries spill and how much; that tells you where raising it would actually help.

maintenance_work_mem applies to VACUUM, CREATE INDEX and ALTER TABLE. It is used by few processes at a time, so it can be much larger than work_mem — a large value materially speeds up vacuum on big tables. Bound it by autovacuum_max_workers × maintenance_work_mem for the vacuum case.

Protecting against the OOM killer

  • Disable memory overcommit for a dedicated database host: vm.overcommit_memory = 2 with an appropriate vm.overcommit_ratio. PostgreSQL then receives allocation failures — which it handles — instead of being killed.
  • Protect the postmaster with a negative oom_score_adj, which the packaged systemd units generally handle.
  • Configure huge pages (huge_pages = try) for large shared_buffers, reducing page-table overhead.
  • In containers, set the memory limit above the sum of shared memory plus expected per-backend usage, and remember the cgroup limit counts page cache.

Connections are memory too

Each connection is a process with its own memory. Hundreds of idle connections consume real memory before executing anything, which is one of the main arguments for a pooler — see Connection Management.