Elasticsearch
Distributed search and analytics engine built on Apache Lucene, with a JSON query DSL and near-real-time indexing.
What it is
A distributed search and analytics engine built on Apache Lucene. Since 2021 it has been licensed under the Elastic License 2.0 and SSPL, with AGPLv3 added in 2024 — the 2021 change is what prompted the OpenSearch fork.
Architecture
An index is split into primary shards, each a self-contained Lucene index, with replicas for redundancy and read capacity. Documents are analysed into terms at index time and stored in an inverted index. Within a shard, data lives in immutable segments that background merges combine. Node roles separate cluster state (master-eligible) from data and coordination.
Best use cases
- Full-text search with relevance ranking, stemming, synonyms and highlighting.
- Log and observability search over large volumes, with time-based rolling indices.
- Faceted navigation, where aggregations over filtered sets are the query.
- Ad-hoc exploration of semi-structured documents.
When not to use it
- As a system of record. No cross-document transactions, delayed visibility after writes, and recovery depends on snapshots.
- For frequently updated documents — an update is a delete plus a full reindex of the document.
- For relational queries; joins are limited and expensive.
- For very high-cardinality aggregations at scale, where a column store is a better tool.
Data model
JSON documents with an explicit mapping defining field types and analysis. The text versus
keyword distinction is central: text is analysed and searchable, keyword is exact and
aggregatable. Dynamic mapping should be strict or disabled in production.
Consistency and transactions
No multi-document transactions. Writes go to the primary shard and replicate; visibility follows a
refresh, by default once per second. Durability depends on index.translog.durability.
Scaling model
Add data nodes and shards. Primary shard count is fixed at index creation, so it must be planned or changed by reindexing. Tiered node roles (hot, warm, cold) match hardware to data age.
Replication
Primary and replica shards, allocated across nodes with awareness of failure domains. Cluster health (green, yellow, red) reports whether primaries and replicas are allocated.
Backup and recovery
Snapshots to a repository — the only backup mechanism. Incremental at the segment level, with snapshot lifecycle policies for scheduling and retention.
Monitoring
_cluster/health, _cat APIs, node stats for JVM heap and thread pool rejections, and search slow
logs. See Search Cluster Monitoring.
Common mistakes
Production checklist
See Search Production Checklist.