kubelet, CRI and the pod sandbox

Kubernetes · Expert · 14 min

Node-level failures, evictions and resource behaviour make sense only with this picture.

The kubelet watches pods bound to its node and drives the container runtime through the Container Runtime Interface (CRI), a gRPC API. Dockershim was removed in 1.24, so nodes run containerd or CRI-O directly; images built with Docker still work, since they're OCI images.

For each pod the kubelet asks the runtime to create a pod sandbox: a set of Linux namespaces (network, IPC, UTS) held by a tiny pause container, and calls the CNI plugin to give the sandbox an IP. Each app container is then created by the runtime via runc (or another OCI runtime such as gVisor or Kata via RuntimeClass) inside that sandbox, sharing its network namespace. That's why containers in a pod share localhost and the IP survives container restarts.

Resources map onto cgroups v2: a hierarchy of kubepods slices per QoS class, then per pod and container. Memory limits become memory.max; CPU requests become cpu.weight and limits cpu.max quotas.

The kubelet also:

  • runs probes and restarts containers with exponential back-off (the CrashLoopBackOff delay),
  • performs node-pressure eviction when memory, disk or PIDs cross thresholds, picking victims by QoS and usage over requests,
  • garbage-collects images and dead containers,
  • reports node status and renews a Lease every 10s as its heartbeat.

crictl talks CRI directly on a node for debugging when the API is no help.

Example

# on a node (e.g. docker exec -it kind-worker bash)
crictl pods | head -3
# POD ID   CREATED  STATE  NAME            NAMESPACE
# 4b1c...  2h       Ready  web-7c6d9-abc   default
crictl ps --pod 4b1c                     # app containers of that sandbox
crictl inspectp 4b1c | grep -m1 '"ip"'   # sandbox IP from CNI

# cgroup of a Burstable pod (cgroup v2, systemd driver)
ls /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/ | head -3
cat /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/*/memory.max | head -1

journalctl -u kubelet --since -10m | grep -i evict

Remember

  • kubelet speaks CRI to containerd/CRI-O; runc creates containers
  • The pause sandbox holds the pod's shared namespaces and IP
  • Requests and limits become cgroup v2 weights and maxima; kubelet evicts under pressure

Watch out for

  • Thinking Kubernetes still needs Docker on nodes; it uses containerd or CRI-O directly.

Check yourself

  1. Why does a pod keep its IP when its app container restarts?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: kubernetes.io/docs/concepts/architecture/cri/