TopolearnMost automation, Lambda code and glue scripts on AWS are written against an SDK, and boto3 is the most common one.
boto3 is the AWS SDK for Python, built on botocore. Create a client with boto3.client("s3"): it exposes every API operation as a snake_case method taking keyword arguments and returning a plain dict. The older resource interface (boto3.resource) offers objects like Bucket, but AWS no longer adds features to it, so prefer clients in new code.
Credentials come from the default credential chain, the same one the CLI uses: explicit arguments, environment variables, the shared config and credentials files (including SSO and assume-role profiles), container credentials (ECS, EKS Pod Identity) and finally the EC2 instance metadata service. Your code should not mention keys at all; the environment decides.
Patterns to know:
- Paginators: list calls return one page (S3 returns up to 1,000 keys).
client.get_paginator("list_objects_v2")iterates all pages for you. - Waiters:
client.get_waiter("instance_running").wait(...)polls until a state is reached. - Errors: service errors raise
botocore.exceptions.ClientError; branch one.response["Error"]["Code"], for exampleNoSuchKey,ThrottlingException,ConditionalCheckFailedException. - Retries: configure with
botocore.config.Config(retries={"mode": "standard"})or"adaptive".
Clients are thread-safe and somewhat expensive to create: build them once, at module level in Lambda, and reuse. Sessions (boto3.Session(profile_name="dev")) are not thread-safe; use one per thread when you need different credentials.
Example
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError
s3 = boto3.client("s3", region_name="eu-west-1",
config=Config(retries={"mode": "standard", "max_attempts": 5}))
s3.create_bucket(Bucket="acme-demo-123",
CreateBucketConfiguration={"LocationConstraint": "eu-west-1"})
for i in range(3):
s3.put_object(Bucket="acme-demo-123", Key=f"logs/{i}.txt", Body=b"hi")
pages = s3.get_paginator("list_objects_v2").paginate(Bucket="acme-demo-123", Prefix="logs/")
print([o["Key"] for page in pages for o in page.get("Contents", [])])
# ['logs/0.txt', 'logs/1.txt', 'logs/2.txt']
try:
s3.head_object(Bucket="acme-demo-123", Key="missing.txt")
except ClientError as e:
print(e.response["Error"]["Code"]) # 404Remember
- Prefer clients; responses are plain dicts
- Never put keys in code; rely on the credential chain
- Use paginators for any list call and waiters for state changes
- Catch ClientError and branch on the error code
Watch out for
- Calling list_objects_v2 once and assuming you got every key.