TopolearnThe same code must authenticate on a laptop, in CI and in production without hard-coded secrets.
Google client libraries use Application Default Credentials (ADC). They look for credentials in this order:
- the file named by the
GOOGLE_APPLICATION_CREDENTIALSenvironment variable; - the user credentials created by
gcloud auth application-default login(a file in your gcloud config directory); - the metadata server, available on Compute Engine, GKE, Cloud Run, Cloud Run functions and Cloud Build, which serves tokens for the attached service account.
So code written as storage.Client() works everywhere: on your laptop it uses your ADC login, in production the attached service account. Note that gcloud auth login (credentials for the CLI) and gcloud auth application-default login (credentials for your code) are separate.
To test with exactly production's permissions, impersonate the service account instead of downloading a key. You need roles/iam.serviceAccountTokenCreator on it; then:
gcloud auth application-default login --impersonate-service-account=orders-api@shop-dev-4821.iam.gserviceaccount.comThe CLI has the same feature per command (--impersonate-service-account) or as a config property (auth/impersonate_service_account).
Libraries also need a quota project for some APIs when using user credentials; gcloud auth application-default set-quota-project fixes the related warning.
Example
# pip install google-cloud-storage
import google.auth
from google.cloud import storage
credentials, project = google.auth.default()
print(type(credentials).__name__, project)
# laptop with ADC login: Credentials shop-dev-4821
# Cloud Run / GCE / GKE: Credentials (compute_engine) shop-dev-4821
client = storage.Client() # same discovery, no key file
for bucket in client.list_buckets(max_results=3):
print(bucket.name)Remember
- ADC order: GOOGLE_APPLICATION_CREDENTIALS, gcloud ADC file, metadata server
- gcloud auth login is for the CLI; application-default login is for your code
- Impersonate a service account (Token Creator) instead of using keys
Watch out for
- Leaving GOOGLE_APPLICATION_CREDENTIALS pointing at an old key file, which silently overrides everything else.