Indices, documents and the REST API

Elasticsearch · Essentials · 8 min

Knowing the core vocabulary and API style lets you read any Elasticsearch example or log query.

Elasticsearch is a search engine you talk to over HTTP with JSON. Data lives in indices (roughly, a table), each holding documents: JSON objects identified by an _id. Every document's original JSON is kept in _source and its fields are indexed so they can be searched.

Under the hood each index is split into shards, each a Lucene index, spread across the nodes of a cluster. You don't address shards directly; any node can receive a request and coordinate it.

The API is resource-oriented:

  • PUT /products creates an index, DELETE /products removes it,
  • PUT /products/_doc/1 stores a document, GET /products/_doc/1 fetches it,
  • GET /products/_search searches, GET /_cluster/health checks the cluster.

Examples in docs use the Kibana Dev Tools console format: a method and path on one line followed by a JSON body. The same requests work with curl or any client library (Python, JavaScript, Java, Go...).

Elasticsearch is near real-time: a newly indexed document becomes searchable after the next refresh, by default within about a second, though GET by id sees it immediately. It is not a primary transactional database: there are no multi-document transactions, so it usually holds a searchable copy of data whose source of truth lives elsewhere.

Example

GET /

GET /_cluster/health
# { "cluster_name": "docker-cluster", "status": "green", "number_of_nodes": 1, ... }

PUT /products

GET /_cat/indices?v&h=health,status,index,pri,rep,docs.count
# health status index    pri rep docs.count
# yellow open   products   1   1          0

# curl equivalent of the health call
# curl -u elastic:$ES_PASSWORD https://localhost:9200/_cluster/health?pretty

Remember

  • Index = collection of JSON documents; each has an _id and _source
  • Everything is a REST call with a JSON body
  • Indices are split into shards spread over nodes
  • Near real-time: searchable after a refresh (about 1 s)

Watch out for

  • Using Elasticsearch as the only copy of critical transactional data.

Check yourself

  1. You index a document and immediately search for it, but it is not found; a second later it is. Why?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.elastic.co/guide/en/elasticsearch/reference/current/documents-indices.html

Builds on

Nothing: this is a foundation.

Unlocks