TopolearnHigh availability and special hardware both depend on putting pods in the right places.
The scheduler filters nodes that can run a pod (enough requested resources, matching constraints) and scores the rest. You influence it with:
nodeSelector: the simplest rule, node labels that must match (kubernetes.io/os: linux,node.kubernetes.io/instance-type, your ownpool: gpu).- Node affinity: richer expressions,
requiredDuringSchedulingIgnoredDuringExecution(hard) orpreferred...with weights (soft). - Pod affinity / anti-affinity: place pods near or away from other pods by label within a
topologyKey(a node, a zone). - Taints and tolerations: a taint on a node (
dedicated=gpu:NoSchedule) repels pods without a matching toleration. Tolerating a taint only allows a pod there; combine it with affinity to also attract it.NoExecutetaints also evict running pods. - topologySpreadConstraints: keep replicas evenly spread across zones and nodes with
maxSkew, the usual tool for HA.
"IgnoredDuringExecution" means rules are checked only when scheduling; a running pod isn't moved if labels change later. The descheduler add-on can rebalance.
PriorityClass sets pod priority: when the cluster is full, a high-priority pending pod can preempt lower-priority ones.
If a pod is Pending, kubectl describe shows messages like 0/5 nodes are available: 3 node(s) had untolerated taint....
Example
spec:
replicas: 6
template:
metadata: {labels: {app: api}}
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector: {matchLabels: {app: api}}
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- {key: pool, operator: In, values: [general]}
tolerations:
- {key: spot, operator: Equal, value: "true", effect: NoSchedule}
# kubectl taint nodes node-7 spot=true:NoScheduleRemember
- nodeSelector/affinity attract; taints repel unless tolerated
- Topology spread keeps replicas across zones
- Rules apply at scheduling time only
Watch out for
- Adding a toleration and expecting pods to go to the tainted nodes: tolerations permit, they don't attract.