Elasticsearch and OpenSearchintermediate

Index Lifecycle Management

Rollover, phase transitions and tiered storage for time-based indices.

2 min readIntermediateUpdated 31 Jul 2026Edit this page

Time-based data — logs, events, metrics — belongs in rolling indices rather than one growing index. Lifecycle management automates the rollover and the eventual deletion.

Why rolling indices

Deletion is dropping an index, which is instant, rather than a delete-by-query that rewrites segments.
Old indices can be moved to cheaper storage without touching the active one.
Old indices can be force-merged and made read-only, reducing their overhead.
Shard count can change over time as volume changes, because each new index is created fresh.

A policy

PUT _ilm/policy/logs-policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": { "max_primary_shard_size": "50gb", "max_age": "1d" },
          "set_priority": { "priority": 100 }
        }
      },
      "warm": {
        "min_age": "2d",
        "actions": {
          "allocate": { "require": { "data": "warm" }, "number_of_replicas": 1 },
          "forcemerge": { "max_num_segments": 1 },
          "set_priority": { "priority": 50 }
        }
      },
      "cold": {
        "min_age": "30d",
        "actions": {
          "allocate": { "require": { "data": "cold" }, "number_of_replicas": 0 },
          "set_priority": { "priority": 0 }
        }
      },
      "delete": {
        "min_age": "90d",
        "actions": { "delete": {} }
      }
    }
  }
}

The equivalent in OpenSearch is Index State Management (ISM), with the same concepts and different API paths.

PUT _index_template/logs-template
{
  "index_patterns": ["logs-*"],
  "template": {
    "settings": {
      "index.lifecycle.name": "logs-policy",
      "index.lifecycle.rollover_alias": "logs",
      "number_of_shards": 3,
      "number_of_replicas": 1
    }
  }
}

Write through the alias; rollover repoints it to the new index automatically.

Rollover only happens when the policy is evaluated

Lifecycle checks run periodically, not continuously, so an index can exceed its rollover threshold before the transition occurs. Size thresholds with headroom, and monitor for indices stuck in a phase:

GET /logs-*/_ilm/explain

Hot-Warm-Cold Architecture

Nodes are labelled by tier, and the policy moves indices between them:

# elasticsearch.yml on a warm node
node.attr.data: warm

Tier	Hardware	Holds
Hot	Fast NVMe, more CPU and heap	Actively written and frequently queried indices
Warm	Larger, slower disks	Recent read-only indices, force-merged
Cold	Cheapest storage, often fewer replicas	Rarely queried history

Verifying it works

GET /_cat/indices/logs-*?v&s=index
GET /logs-*/_ilm/explain?only_errors=true

Alert on lifecycle errors. A policy that silently stops progressing produces an index that never rolls over, never deletes, and eventually fills the cluster.