Shards, replicas and cluster health

Elasticsearch · Advanced · 13 min

Shard layout decides how an index scales, survives node failures and performs, and is hard to change later.

An index is divided into primary shards; each document lives in exactly one, chosen by routing (by default a hash of _id). Each primary can have replica shards: full copies on other nodes that provide failover and serve searches too.

  • number_of_shards is fixed at creation (changing it means _split, _shrink or reindexing); the default is 1,
  • number_of_replicas can be changed any time; the default is 1,
  • a replica is never placed on the same node as its primary.

Cluster health summarises allocation: green means all primaries and replicas are assigned; yellow means all primaries are assigned but some replicas aren't (a single-node cluster with replicas is always yellow); red means some primary is unassigned, so some data is unavailable. GET _cluster/allocation/explain tells you why a shard isn't assigned.

Sizing guidance: aim for shards of roughly 10 to 50 GB and avoid thousands of tiny shards, since every shard costs heap, file handles and coordination. More shards give more parallelism for writes; more replicas give more read throughput and resilience but multiply storage.

Nodes have roles: master-eligible nodes manage cluster state (use three for resilience), data nodes hold shards (possibly tiered hot/warm/cold), ingest nodes run pipelines, and any node can coordinate requests.

Example

PUT /logs-2026.10
{ "settings": { "number_of_shards": 3, "number_of_replicas": 1 } }

PUT /logs-2026.10/_settings
{ "number_of_replicas": 2 }

GET /_cat/shards/logs-2026.10?v&h=index,shard,prirep,state,docs,store,node
# index        shard prirep state   docs  store node
# logs-2026.10 0     p      STARTED 10231 4.1mb node-1
# logs-2026.10 0     r      STARTED 10231 4.1mb node-2
# ...

GET /_cluster/health?filter_path=status,unassigned_shards
# { "status": "yellow", "unassigned_shards": 3 }
GET /_cluster/allocation/explain

Remember

  • Primary shard count is fixed at creation; replicas can change
  • Green: all assigned; yellow: replicas missing; red: a primary missing
  • Aim for roughly 10-50 GB shards; avoid many tiny ones
  • Use three master-eligible nodes for a resilient cluster

Watch out for

  • Creating many indices with many shards each (oversharding), exhausting heap with small shards.

Check yourself

  1. A one-node cluster has an index with 1 replica. What health does it report?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.elastic.co/guide/en/elasticsearch/reference/current/scalability.html