DNS and service discovery

Kubernetes · Practical · 8 min

Every service-to-service call relies on cluster DNS; its search paths explain cross-namespace surprises.

Clusters run CoreDNS (in kube-system) and every pod's /etc/resolv.conf points at it. Each Service gets a name:

<service>.<namespace>.svc.cluster.local

So from a pod in namespace shop, these all reach the api Service in shop: api, api.shop, api.shop.svc, api.shop.svc.cluster.local. From another namespace you need at least api.shop. This works because resolv.conf has search domains (shop.svc.cluster.local svc.cluster.local cluster.local) and ndots:5.

Record types:

  • ClusterIP Service: an A/AAAA record with the virtual IP.
  • Headless Service: A records for each ready pod IP, and for StatefulSet pods stable names like db-0.db.shop.svc.cluster.local.
  • Named ports get SRV records.

The ndots:5 default means a name with fewer than five dots, like api.github.com, is first tried with each search suffix, causing several wasted lookups. For hot external calls use a trailing dot (api.github.com.) or lower ndots in the pod's dnsConfig.

Kubernetes also injects <SERVICE>_SERVICE_HOST/_PORT environment variables for Services that existed when the pod started; DNS is preferred because it doesn't depend on creation order.

Example

kubectl -n shop run dns --rm -it --image=busybox:1.37 -- sh
/ # cat /etc/resolv.conf
# search shop.svc.cluster.local svc.cluster.local cluster.local
# nameserver 10.96.0.10
# options ndots:5
/ # nslookup api                      # same namespace
# Name:   api.shop.svc.cluster.local
# Address: 10.96.112.7
/ # nslookup postgres.data            # Service postgres in namespace data
/ # nslookup db-0.db.data.svc.cluster.local   # one StatefulSet pod

Remember

  • Service DNS: name.namespace.svc.cluster.local
  • Short names work only within the same namespace
  • Headless Services resolve to pod IPs; ndots:5 adds lookups for external names

Watch out for

  • Calling http://api from another namespace and getting NXDOMAIN; use api.<namespace>.

Check yourself

  1. A pod in namespace web calls http://db:5432, but the db Service lives in namespace data. What happens?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: kubernetes.io/docs/concepts/services-networking/dns-pod-service/