CRDs and operators

Kubernetes · Expert · 15 min

Most platform tooling (cert-manager, Argo CD, database operators) is CRDs plus controllers.

A CustomResourceDefinition adds a new resource type to the API server: a group, a kind, versions, and an OpenAPI v3 schema for validation. Once applied, kubectl get certificates works like a built-in resource, with RBAC, watch and storage in etcd for free.

CRD features worth knowing:

  • Structural schemas with defaults, enums and x-kubernetes-validations CEL rules (e.g. self.minReplicas <= self.maxReplicas) validated by the API server.
  • The status subresource separates spec (users) from status (controller) writes, and scale enables HPA and kubectl scale.
  • additionalPrinterColumns for kubectl get output.
  • Several versions with one storage version; converting between them needs a conversion webhook when schemas differ.

A CRD alone stores data. An operator is a controller that reconciles those custom resources into real things: a PostgresCluster object becomes StatefulSets, Services, backups and failover logic. It encodes the operational knowledge a human SRE would apply.

Operators are written with controller-runtime via Kubebuilder or Operator SDK in Go, or with frameworks like kopf (Python). The reconcile function should be idempotent, set owner references on what it creates, use finalizers for external cleanup and report conditions in status.

Before writing one, check whether a ConfigMap plus a Job, or an existing operator, already solves the problem.

Example

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata: {name: backups.ops.example.com}
spec:
  group: ops.example.com
  names: {kind: Backup, plural: backups, singular: backup}
  scope: Namespaced
  versions:
    - name: v1
      served: true
      storage: true
      subresources: {status: {}}
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              required: [database, schedule]
              properties:
                database: {type: string}
                schedule: {type: string}
                keep: {type: integer, minimum: 1, default: 7}
# kubectl get backups   (a controller must still act on them)

Remember

  • A CRD adds an API type with schema validation and CEL rules
  • An operator is a controller reconciling custom resources
  • Use status subresource, owner refs, finalizers and conditions

Watch out for

  • Writing an operator for something a Helm chart or CronJob would handle.

Check yourself

  1. You apply a CRD and create a custom resource, but nothing happens in the cluster. Why?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/

Builds on

Unlocks

Nothing builds on this yet.