Indexes and explain

MongoDB · Practical · 12 min

Without the right index, queries that are instant in development become full collection scans in production.

Without an index MongoDB answers a query with a collection scan (COLLSCAN): it reads every document. An index is a sorted B-tree of field values pointing to documents, so equality and range lookups and sorts on that field become cheap index scans (IXSCAN).

Every collection has a unique index on _id. Create others with createIndex({ email: 1 }); the 1 or -1 is the sort direction, which only matters for multi-field indexes. Options include:

  • unique: true to reject duplicates (inserts fail with error 11000),
  • partialFilterExpression to index only some documents, such as { deletedAt: null },
  • expireAfterSeconds for TTL indexes, sparse for documents that have the field.

Indexing an array field creates a multikey index with one entry per element, so { tags: "sale" } is fast too.

Check what a query does with explain("executionStats"). Compare nReturned with totalDocsExamined and totalKeysExamined: a good plan examines about as many keys as it returns. Seeing COLLSCAN on a hot query, or examined counts far larger than returned, means you need a better index.

Indexes aren't free: each one costs RAM, disk and work on every insert and update. Index for the queries you actually run, and drop unused ones ($indexStats shows usage).

Example

for (let i = 0; i < 50000; i++) db.users.insertOne({ email: `u${i}@x.io`, age: i % 90 })

let s = db.users.find({ email: "u4242@x.io" }).explain("executionStats")
s.queryPlanner.winningPlan.stage           // 'COLLSCAN'
s.executionStats.totalDocsExamined         // 50000

db.users.createIndex({ email: 1 }, { unique: true })   // 'email_1'
s = db.users.find({ email: "u4242@x.io" }).explain("executionStats")
s.executionStats.totalKeysExamined         // 1
s.executionStats.totalDocsExamined         // 1   (plan uses IXSCAN)

db.users.insertOne({ email: "u1@x.io" })   // E11000 duplicate key error
db.users.getIndexes().map(i => i.name)     // [ '_id_', 'email_1' ]

Remember

  • No index means COLLSCAN: every document is read
  • explain("executionStats"): compare keys/docs examined with nReturned
  • unique, partial and TTL are index options; array fields get multikey indexes
  • Every index slows writes and uses memory: index for real queries

Watch out for

  • Creating an index for every field just in case: write throughput and RAM suffer.

Check yourself

  1. A query returns 10 documents but explain shows totalDocsExamined: 2000000. What does that suggest?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.mongodb.com/docs/manual/indexes/