TopolearnWhen the wrong identity is used in production, or credentials expire mid-job, the fix requires knowing the chain precisely.
Credentials passed explicitly to client() or Session() always win. Otherwise botocore's resolver tries providers in order and uses the first that returns credentials:
- Environment variables (
AWS_ACCESS_KEY_ID, ...). Skipped when you passprofile_name=to a Session explicitly. - Profile-based providers for the active profile: assume role (
role_arn), web identity (web_identity_token_file), IAM Identity Center/SSO (sso_session), the shared credentials file, console login,credential_process(an external command printing JSON), and static keys in the config file. - Legacy files (
/etc/boto.cfg,~/.boto). - Container credentials (
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI/_FULL_URI, used by ECS tasks and EKS Pod Identity). - Instance metadata (IMDS) on EC2.
session.get_credentials().method tells you which provider won: env, assume-role, sso, shared-credentials-file, container-role, iam-role.
Most providers return RefreshableCredentials: botocore checks expiry before each request and refreshes ahead of time (advisory and mandatory refresh windows), thread-safely. Assume-role and SSO results are cached on disk under ~/.aws/cli/cache and ~/.aws/sso/cache.
Classic surprises: stale AWS_ACCESS_KEY_ID in a shell overriding the instance role; a container without the metadata hop limit raised (IMDSv2 on Docker); a frozen credentials object (get_frozen_credentials()) used past expiry. Debug with boto3.set_stream_logger("botocore.credentials").
Example
import logging, os
import boto3
os.environ["AWS_ACCESS_KEY_ID"] = "AKIAEXAMPLE" # simulate a stray env var
os.environ["AWS_SECRET_ACCESS_KEY"] = "secret"
creds = boto3.Session().get_credentials()
print(creds.method) # env: env vars beat profiles and roles
frozen = creds.get_frozen_credentials() # a snapshot: never refreshes
print(frozen.access_key) # AKIAEXAMPLE
# See which providers are tried, in order, and which one wins:
boto3.set_stream_logger("botocore.credentials", logging.DEBUG)
boto3.Session().get_credentials()
# ... Looking for credentials via: env
# ... Found credentials in environment variables.
# credential_process example (~/.aws/config):
# [profile vault]
# credential_process = /usr/local/bin/vault-aws-creds --role deployRemember
- Explicit credentials > env vars > profile providers > container > instance metadata
- profile_name= on a Session skips env var credentials
- creds.method reveals which provider supplied credentials
- Refreshable credentials renew themselves; frozen snapshots don't
Watch out for
- A leftover AWS_ACCESS_KEY_ID in the environment silently overriding the role you expected to use.