Indexing, getting and updating documents

Elasticsearch · Essentials · 9 min

Every search starts with getting documents in, and the document APIs are the building blocks of syncing data.

PUT /index/_doc/<id> creates or replaces the document with that id; POST /index/_doc lets Elasticsearch generate an id. PUT /index/_create/<id> fails with a 409 if the id exists, which is useful for idempotent inserts. If the index doesn't exist, it is created with default settings and dynamic mapping guesses field types.

GET /index/_doc/<id> returns the document's _source plus metadata: _version, _seq_no and _primary_term. Unlike search, get by id is real-time.

Documents are immutable in Lucene, so an "update" is a reindex: POST /index/_update/<id> with { "doc": { ... } } merges fields into the stored source and writes a new version. Scripted updates ("script": { "source": "ctx._source.stock -= params.n" }) and doc_as_upsert are available too. DELETE /index/_doc/<id> removes one document.

For safe concurrent writes use optimistic concurrency control: read _seq_no and _primary_term, then write with ?if_seq_no=..&if_primary_term=..; if someone changed the document meanwhile you get a 409 conflict and can retry.

Add ?refresh=wait_for in tests when you need the change visible to search before the call returns, but avoid it in high-throughput code.

Example

PUT /products/_doc/1
{ "name": "Blue pen", "price": 1.5, "tags": ["office"], "stock": 10 }
# { "_id": "1", "_version": 1, "result": "created", "_seq_no": 0, "_primary_term": 1 }

POST /products/_update/1
{ "doc": { "price": 1.2 } }
# { "_version": 2, "result": "updated", ... }

POST /products/_update/1
{ "script": { "source": "ctx._source.stock -= params.n", "params": { "n": 3 } } }

GET /products/_doc/1
# { "_version": 3, "_seq_no": 2, "found": true,
#   "_source": { "name": "Blue pen", "price": 1.2, "tags": ["office"], "stock": 7 } }

PUT /products/_doc/1?if_seq_no=1&if_primary_term=1
{ "name": "Stale write" }
# 409 version_conflict_engine_exception

Remember

  • PUT with an id creates or replaces; POST without id generates one
  • _update merges a partial doc or runs a script, creating a new version
  • GET by id is real-time; search waits for refresh
  • if_seq_no + if_primary_term give optimistic concurrency

Watch out for

  • Using PUT /index/_doc/id with partial fields and wiping the rest: PUT replaces the whole document.

Check yourself

  1. What does PUT /products/_doc/1 with body { "price": 2 } do to an existing document with name and price?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html