2

In Kubernetes, if I run a Service Foo on every Node, and also Service Bar on every Node, i.e. with DaemonSets. If a Pod from Service Foo needs to make a request to Service Bar, does Kubernetes networking magic optimise things so communication will be done within the same Node, i.e. without a external network call? Or will a random Pod Bar be picked meaning usually Bar will be on a different Node, so there will be a network hop?

Context to question: I'm thinking to run CoreDNS as a Daemonset, because with the cache plugin, I'm hoping most DNS queries will not need to use the network as there will always be a CoreDNS Pod co-located within the same Node and most requests will be cached. Currently we only run 3 CoreDNS pods and we have 12 Nodes and we see some DNS failures at times of network traffic surges - particularly during a deployment when lot of logging is done, which have to be shipped across the network to ElasticSearch.

In case specifics matter:

  • The CoreDNS service runs as clusterIP: 172.20.0.10
  • The services that use CoreDNS run as type: NodePort
  • The Kubernetes cluster is AWS EKS, v1.14.9. Networking is done with the AWS VPC CNI.
Tom
  • 151
  • 7

2 Answers2

3

No, Kubernetes will load-balance requests to a ClusterIP across all Pods with matching labels (and passing readiness checks). It won't keep traffic within the same Node.

Q: So how does the NodeLocal DNS Cache achieve to keep DNS requests local to the Node?

A: When using the NodeLocal DNS Cache, it is necessary to pass a extra arg to the Kubelet of each Node (e.g. --kubelet-extra-args "--cluster-dns=169.254.20.10 ..."). This changes the resolv.conf of every container in every Pod to use DNS resolver on the Node, instead of a clusterIP. The NodeLocal Cache runs on every Node (as a DaemonSet) and uses a arg (e.g. "-localip", "169.254.20.10") to run on the specific IP set in the resolv.conf of all Pods. The NodeLocal DNS Cache Pod also needs to be configured with hostNetwork: true.

Tom
  • 151
  • 7
2

This looks to be answered on offcail docs here

NodeLocal DNSCache improves Cluster DNS performance by running a dns caching agent on cluster nodes as a DaemonSet.

With this architecture, Pods will reach out to the dns caching agent running on the same node, thereby avoiding iptables DNAT rules and connection tracking.

Having a local cache will help improve the latency.

[]

Image Reference : https://kubernetes.io/docs/tasks/administer-cluster/nodelocaldns/#architecture-diagram

Community
  • 1
DT.
  • 260
  • 2
  • 10