DynamoDB with boto3

boto3 · Practical · 14 min

DynamoDB is the default serverless database on AWS, and boto3's Table API has several Python-specific gotchas.

For DynamoDB, the resource API is still the most convenient: table = boto3.resource("dynamodb").Table("orders"). It converts between Python types and DynamoDB's typed JSON ({"S": "x"}, {"N": "1"}) for you; the client API makes you write those type descriptors by hand.

Core operations:

  • table.put_item(Item={...}) writes (replacing any item with the same key).
  • table.get_item(Key={"pk": ..., "sk": ...}) returns a dict with "Item" only if found: use .get("Item").
  • table.update_item(Key=..., UpdateExpression="SET #s = :s", ExpressionAttributeNames=..., ExpressionAttributeValues=...).
  • table.query(KeyConditionExpression=Key("pk").eq("user#1") & Key("sk").begins_with("order#")) fetches items of one partition efficiently. scan reads the whole table: avoid it on large tables.

Conditions come from boto3.dynamodb.conditions (Key, Attr). Conditional writes like ConditionExpression=Attr("pk").not_exists() implement "create only if absent" atomically; failures raise ConditionalCheckFailedException.

Numbers are `Decimal`: DynamoDB numbers come back as decimal.Decimal, and passing a Python float raises TypeError. Convert with Decimal(str(x)).

Query results are paginated by size (1 MB per call); follow LastEvaluatedKey with ExclusiveStartKey, or use the client's paginator. Design keys around your access patterns before writing code.

Example

from decimal import Decimal
import boto3
from boto3.dynamodb.conditions import Key, Attr

ddb = boto3.resource("dynamodb", region_name="us-east-1")
table = ddb.create_table(
    TableName="orders", BillingMode="PAY_PER_REQUEST",
    KeySchema=[{"AttributeName": "pk", "KeyType": "HASH"}, {"AttributeName": "sk", "KeyType": "RANGE"}],
    AttributeDefinitions=[{"AttributeName": "pk", "AttributeType": "S"},
                          {"AttributeName": "sk", "AttributeType": "S"}])
table.wait_until_exists()

for n, total in [(1, "19.90"), (2, "5.00"), (3, "42.10")]:
    table.put_item(Item={"pk": "user#1", "sk": f"order#{n:03}", "total": Decimal(total)})

print(table.get_item(Key={"pk": "user#1", "sk": "order#002"})["Item"]["total"])  # 5.00
print(table.get_item(Key={"pk": "user#1", "sk": "order#999"}).get("Item"))      # None

resp = table.query(KeyConditionExpression=Key("pk").eq("user#1") & Key("sk").begins_with("order#"),
                   FilterExpression=Attr("total").gt(Decimal("10")))
print([i["sk"] for i in resp["Items"]])               # ['order#001', 'order#003']
try:
    table.put_item(Item={"pk": "user#1", "sk": "order#001"}, ConditionExpression=Attr("pk").not_exists())
except table.meta.client.exceptions.ConditionalCheckFailedException:
    print("already exists")                            # already exists

Remember

  • The Table resource converts Python values to DynamoDB types
  • get_item has no 'Item' key when nothing matches
  • query by partition key with Key conditions; avoid scan on big tables
  • Numbers are Decimal; floats raise TypeError

Watch out for

  • Passing a float like 19.9 in an item and getting 'Float types are not supported'.

Check yourself

  1. What does table.get_item(Key=...) return when no item matches?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html