How Helm applies changes and stores releases

Helm · Expert · 12 min

Stuck releases, adoption errors and fights with other controllers all come from these mechanics.

Release storage: each revision is a Secret named sh.helm.release.v1.<release>.v<revision> in the release namespace, labelled owner=helm. Its data is the release (chart, values, rendered manifest, status) gzipped and base64-encoded. Anyone who can read Secrets in that namespace can read your values, including passwords. The storage driver can be switched (HELM_DRIVER=configmap, sql), and --history-max keeps it bounded.

Ownership: Helm labels and annotates what it creates (app.kubernetes.io/managed-by: Helm, meta.helm.sh/release-name, meta.helm.sh/release-namespace). Installing a chart over an existing object without these fails with an "invalid ownership metadata" error; adding the annotations by hand lets Helm adopt it. helm.sh/resource-policy: keep leaves an object behind on uninstall (useful for PVCs or CRDs).

Applying changes: Helm 3 computes a three-way merge between the previous release manifest, the new manifest and the live object, so changes made by others to fields Helm never set survive. Helm 4 uses Kubernetes server-side apply for new releases, with field ownership tracked by the API server; conflicts with other managers (an HPA owning replicas) are then explicit.

Drift: Helm only compares against what it rendered; it does not continuously correct manual edits. GitOps controllers fill that gap.

CRDs in a chart's crds/ folder are installed once and never upgraded or deleted by Helm, on purpose; manage CRD upgrades separately.

Example

kubectl -n shop get secret sh.helm.release.v1.shop.v7 -o jsonpath='{.data.release}' \
  | base64 -d | base64 -d | gunzip | jq '{name, version, status: .info.status, values: .config}'
# {"name":"shop","version":7,"status":"deployed","values":{"image":{"tag":"1.5.0"}}}

# adopt an existing object into a release
kubectl -n shop annotate svc shop-web meta.helm.sh/release-name=shop \
  meta.helm.sh/release-namespace=shop
kubectl -n shop label svc shop-web app.kubernetes.io/managed-by=Helm

kubectl get deploy shop-web -n shop --show-managed-fields -o yaml | grep 'manager:'

Remember

  • Release revisions are gzipped Secrets: protect read access
  • Ownership annotations decide what Helm manages; resource-policy keep
  • Helm 3 three-way merge vs Helm 4 server-side apply; CRDs in crds/ are never upgraded

Watch out for

  • Assuming Helm will reconcile manual kubectl edits back to the chart's state on its own.

Check yourself

  1. Why does upgrading a chart not update the CRDs in its crds/ directory?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: helm.sh/docs/topics/advanced/