Sharding and shard keys

MongoDB · Expert · 15 min

When one replica set can't hold the data or the write load, the shard key decides whether the cluster scales or hot-spots.

A sharded cluster splits a collection across several shards (each a replica set). Applications connect to mongos routers, which use metadata on the config server replica set to send each operation where the data lives.

Data is partitioned by the shard key, one or more indexed fields present in every document. Key space is divided into contiguous ranges (historically called chunks), and the balancer moves ranges between shards to even out data.

  • Ranged sharding keeps nearby key values together, good for range queries but a monotonically increasing key like a timestamp or ObjectId sends every insert to the last range: a hot shard.
  • Hashed sharding spreads inserts evenly but turns range queries on the key into scatter-gather.

A good key has high cardinality, low frequency (no single value dominating) and non-monotonic growth, and appears in most queries. Queries that include the key are targeted to one or few shards; queries without it are broadcast to all shards (scatter-gather), and their latency is that of the slowest shard.

Compound keys such as { tenantId: 1, orderId: 1 } are common: queries per tenant are targeted while one big tenant can still spread across ranges. You can refine a key by adding suffix fields, or change it entirely with reshardCollection (5.0+), which copies the data, so choose carefully up front.

Example

// via mongos
sh.enableSharding("shop")                 // optional since 6.0, harmless
db.orders.createIndex({ tenantId: 1, orderId: 1 })
sh.shardCollection("shop.orders", { tenantId: 1, orderId: 1 })

// targeted: includes the shard key prefix -> one or few shards
db.orders.find({ tenantId: "acme", status: "open" })
// scatter-gather: no shard key -> every shard is queried
db.orders.find({ status: "open" })

// hashed alternative for an append-heavy event stream
sh.shardCollection("shop.events", { deviceId: "hashed" })
db.orders.getShardDistribution()          // data and docs per shard

Remember

  • mongos routes, config servers store metadata, shards store data
  • The shard key determines distribution and query targeting
  • Monotonic ranged keys hot-spot one shard; hashed keys spread writes
  • Queries without the shard key fan out to all shards

Watch out for

  • Sharding on a low-cardinality field such as country or status: ranges can't be split and one shard fills up.

Check yourself

  1. A collection is range-sharded on { createdAt: 1 } and receives heavy inserts. What problem appears?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.mongodb.com/docs/manual/sharding/