Hooks and chart tests

Helm · Advanced · 10 min

Database migrations before an upgrade and smoke tests after it are common release steps.

A hook is a normal template with the annotation helm.sh/hook, which makes Helm run it at a lifecycle point instead of applying it with the rest of the release:

  • pre-install, post-install, pre-upgrade, post-upgrade, pre-delete, post-delete, pre-rollback, post-rollback, and test.

Helm waits for hook Jobs and Pods to complete before continuing; a failed hook fails the operation. Order several hooks with helm.sh/hook-weight (lower runs first). Hook resources are not part of the release, so clean them up with helm.sh/hook-delete-policy: before-hook-creation (delete the old one before re-running, the usual choice), hook-succeeded or hook-failed.

Typical use: a pre-upgrade Job runs schema migrations with the new image before new pods start. Make migrations backward compatible, because old pods still run during the rollout, and a rollback won't reverse them.

Caveat: pre-install hooks run before the release's ConfigMaps and Secrets exist, so a hook that needs them must create its own or run post-install.

Chart tests are hooks with helm.sh/hook: test, usually a pod that curls the service. helm test shop -n shop runs them on demand, an easy smoke test after deploy in CI.

GitOps tools map Helm hooks to their own mechanisms, sometimes with differences; check before relying on exotic hook combinations.

Example

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ .Release.Name }}-migrate
  annotations:
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-weight": "0"
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  backoffLimit: 1
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          args: ["migrate", "--to", "latest"]
          envFrom: [{secretRef: {name: {{ .Values.db.secretName }}}}]
# helm upgrade shop ./shop --wait   -> Job runs and must succeed first
# helm test shop                    -> runs templates annotated "helm.sh/hook": test

Remember

  • helm.sh/hook annotations run resources at lifecycle points
  • Hooks aren't release resources: set a delete policy
  • helm test runs test hooks for post-deploy smoke checks

Watch out for

  • Migrations that break the old version still running during the rollout.

Check yourself

  1. Why add hook-delete-policy: before-hook-creation to a migration Job hook?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: helm.sh/docs/topics/charts_hooks/