S3 security, encryption and presigned URLs

Amazon Web Services · Practical · 14 min

Misconfigured buckets are among the most common cloud data leaks; the controls are simple once you know them.

New buckets are private by default: Block Public Access is on and Object Ownership is set to bucket owner enforced, which disables ACLs so that access is governed only by policies. Keep it that way, and enable Block Public Access at the account level too. Public websites should sit behind CloudFront with Origin Access Control instead.

Bucket policies are resource-based policies for cross-account access and guardrails, for example denying any request without TLS (aws:SecureTransport false) or from outside your organization.

Encryption at rest is always on: since 2023 every new object is encrypted with SSE-S3 by default. Choose SSE-KMS with a customer managed key when you need key-level access control and audit; enable S3 Bucket Keys to cut KMS costs.

Presigned URLs grant temporary access to one object without making anything public. The SDK signs a URL with the caller's credentials and an expiry; whoever has the URL can GET (or PUT) that object until it expires. The URL can't outlive the credentials that signed it, so a URL signed with a role session stops working when the session expires. Use presigned PUTs to let browsers upload directly to S3.

Further protections:

  • Versioning plus Object Lock (write-once-read-many retention) for ransomware-proof backups.
  • Replication to another Region or account.
  • Access points to give each application its own policy and network controls on a shared bucket.
  • CloudTrail data events or server access logs to see who read what.

Example

import json, boto3
from botocore.config import Config

s3 = boto3.client("s3", region_name="eu-west-1", config=Config(signature_version="s3v4"))
bucket = "acme-private-123"
s3.create_bucket(Bucket=bucket, CreateBucketConfiguration={"LocationConstraint": "eu-west-1"})
s3.put_public_access_block(Bucket=bucket, PublicAccessBlockConfiguration={
    "BlockPublicAcls": True, "IgnorePublicAcls": True,
    "BlockPublicPolicy": True, "RestrictPublicBuckets": True})

tls_only = {"Version": "2012-10-17", "Statement": [{
    "Sid": "DenyInsecureTransport", "Effect": "Deny", "Principal": "*", "Action": "s3:*",
    "Resource": [f"arn:aws:s3:::{bucket}", f"arn:aws:s3:::{bucket}/*"],
    "Condition": {"Bool": {"aws:SecureTransport": "false"}}}]}
s3.put_bucket_policy(Bucket=bucket, Policy=json.dumps(tls_only))

s3.put_object(Bucket=bucket, Key="invoices/42.pdf", Body=b"%PDF-1.7 ...")
url = s3.generate_presigned_url("get_object", ExpiresIn=900,
                                Params={"Bucket": bucket, "Key": "invoices/42.pdf"})
print("X-Amz-Expires=900" in url)   # True: valid for 15 minutes

Remember

  • Block Public Access on, ACLs disabled, policies for all access control
  • Every object is encrypted at rest; use SSE-KMS for key-level control
  • Presigned URLs share one object temporarily, limited by the signer's credentials
  • Versioning, Object Lock and replication protect against deletion

Watch out for

  • Generating a presigned URL for 7 days from a Lambda role session that expires in an hour.

Check yourself

  1. A user needs to download one private object for the next 10 minutes. Best approach?
  2. Why do new buckets ignore ACLs by default?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html