TopolearnBatching cuts round trips for bulk loads and fan-out reads, but it has rules that cause silent data loss if ignored.
BatchWriteItem sends up to 25 put or delete requests (no updates) across one or more tables in one call; BatchGetItem fetches up to 100 items by full primary key. Both are capped at 16 MB per call.
A batch is not a transaction: each item succeeds or fails independently, and there are no conditions. When DynamoDB is throttling or partially busy, it returns the leftovers in UnprocessedItems (writes) or UnprocessedKeys (reads). You must resend those, ideally with exponential backoff. Code that ignores them loses writes without any exception.
In Python, table.batch_writer() handles this for you: it buffers puts and deletes, sends them in groups of 25, and retries unprocessed items automatically. overwrite_by_pkeys=[...] deduplicates repeated keys in the buffer (a batch may not contain the same key twice).
Batches save network round trips, not capacity: each item still costs the same read or write units as an individual call. BatchGetItem reads are eventually consistent unless you set ConsistentRead per table, and results come back in no particular order.
For loading large datasets into a new table, Import from S3 (CSV, DynamoDB JSON or Ion) is faster and consumes no write capacity.
Example
import boto3
ddb = boto3.resource("dynamodb")
table = ddb.Table("Orders")
with table.batch_writer() as batch: # chunks of 25, retries leftovers
for n in range(60):
batch.put_item(Item={"customer_id": "c7", "order_id": f"o{n:03}", "n": n})
keys = [{"customer_id": "c7", "order_id": f"o{n:03}"} for n in (1, 2, 3)]
resp = ddb.batch_get_item(RequestItems={"Orders": {"Keys": keys}})
print(sorted(int(i["n"]) for i in resp["Responses"]["Orders"])) # [1, 2, 3]
print(resp["UnprocessedKeys"]) # {} -> otherwise resend theseRemember
- BatchWriteItem: up to 25 puts/deletes; BatchGetItem: up to 100 gets
- Batches are not atomic; retry UnprocessedItems/UnprocessedKeys
batch_writer()batches and retries for you- Batching saves round trips, not capacity units
Watch out for
- Calling the low-level batch_write_item and never checking UnprocessedItems.