Partitions, hot keys and adaptive capacity

Amazon DynamoDB · Expert · 15 min

Hot partitions are the main cause of throttling at scale, and fixing them requires knowing how data is physically placed.

Internally a table is split into partitions, each stored on SSDs and replicated across three AZs. The partition key's hash places each item. A partition holds up to about 10 GB and serves up to 3,000 read units and 1,000 write units per second. These limits apply per partition regardless of how much capacity the table has overall.

So a table can have plenty of total throughput and still throttle if traffic concentrates on one key, a hot key. DynamoDB mitigates this in several ways:

  • adaptive capacity instantly lends unused throughput to busy partitions, up to the per-partition maximums
  • split for heat: a partition that stays hot is split, spreading its key range across more partitions (this cannot spread a single partition key value that is itself too hot, unless the sort key range can be divided)
  • partitions are added as data grows or provisioned throughput increases, and are never merged back

What this means for design: a single partition key value has a ceiling of about 1,000 writes/s, so very hot values need write sharding or caching. CloudWatch Contributor Insights shows the most accessed and throttled keys. Throttled requests in metrics such as ThrottledRequests with low overall consumption point to a key-distribution problem, not a capacity problem.

LSIs pin an item collection to one partition, hence their 10 GB limit.

Remember

  • Each partition: about 10 GB, 3,000 RCU and 1,000 WCU per second
  • One hot partition key can throttle while the table has spare capacity
  • Adaptive capacity and split for heat help, but cannot fix one ultra-hot key
  • Use Contributor Insights to find hot keys; shard or cache them

Watch out for

  • Raising table capacity to cure throttling caused by a single hot key.

Check yourself

  1. An on-demand table throttles writes while total consumption is low. Most likely cause?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html