TopolearnMost incidents are one of a handful of failure states, each with a known first check.
Start from the pod's status and follow the matching path.
- Pending: not scheduled.
describeshows why: insufficient CPU/memory (requests too big or cluster full), untolerated taints, affinity mismatch, or an unbound PVC (wrong StorageClass, zone). - ContainerCreating for long: volume attach or mount problems, missing ConfigMap/Secret, CNI issues. Check events.
- ErrImagePull / ImagePullBackOff: wrong image name or tag, private registry without
imagePullSecrets, or registry rate limits. - CrashLoopBackOff: the container keeps exiting. Read
logs --previousand the last state's exit code: 1 is an app error, 137 is SIGKILL (oftenOOMKilled), 143 is SIGTERM, 127 is command not found (wrongcommand/args). - Running but 0/1 READY: readiness probe failing; probe events show the HTTP status or timeout.
- Running and ready but unreachable: check the Service's endpoints (
kubectl get endpointslices), selector vs labels,targetPort, NetworkPolicies, then DNS from a debug pod.
Habits: look at events first, change one thing at a time, and check the controller (Deployment, ReplicaSet) too, as some errors, such as quota exceeded, appear only on the ReplicaSet because the pod was never created.
Example
kubectl get pods -o wide
kubectl describe pod api-7d9-k2 | tail -20 # events
kubectl get pod api-7d9-k2 -o jsonpath='{.status.containerStatuses[0].lastState}'
# {"terminated":{"exitCode":137,"reason":"OOMKilled",...}}
kubectl logs api-7d9-k2 --previous | tail
kubectl describe rs -l app=api | grep -A3 Events # e.g. exceeded quota
kubectl get endpointslices -l kubernetes.io/service-name=api
kubectl run net --rm -it --image=nicolaka/netshoot -- curl -sv http://api.shopRemember
- Status tells you which path to follow
- Exit codes: 137 SIGKILL/OOM, 143 SIGTERM, 127 command not found
- Check endpoints, selectors and policies when ready pods are unreachable
Watch out for
- Restarting things before reading events and previous logs, which destroys the evidence.