Should a service cache DNS itself or rely on a local resolver?

In Kubernetes, prefer a node-local caching resolver when DNS behavior matters operationally, and keep application-level DNS caches short or absent unless you explicitly want stale-answer tolerance. Outside Kubernetes, evaluate a local caching resolver when you need shared cache behavior or resolver telemetry; the node-local deployment guidance below applies specifically to Kubernetes. The real choice is not lookup speed in the happy path. It is which layer controls recovery when records change, when NXDOMAIN was cached, or when your recursive resolver starts timing out.

What does each layer actually do?

The resolution path usually runs through an application or runtime cache, the stub resolver, a recursive resolver, and then authoritative servers. RFC 1035 describes cached DNS data as temporary and assigns distribution and name-server-failure handling to resolvers. That makes per-process application caches a policy choice above the DNS resolver layer, not the mechanism the RFC focuses on.

On Linux/glibc, resolv.conf(5) shows the stub resolver is mainly query routing and retry behavior: it queries listed nameservers, usually in order, with configurable timeout and attempt counts. That distinction matters. The Linux/glibc stub is not a shared node cache. If many processes all miss at once during an upstream resolver incident, they can each pay retry cost separately.

When is application-level DNS caching the wrong boundary?

For names that can move and where fast failover matters, long application caching is often the wrong boundary. A database failover alias, service-discovery name, or external API endpoint may change while the process keeps using an old answer. In that case, your recovery time can become the process cache lifetime, not the DNS TTL or the resolver refresh path.

Negative caching makes this sharper. RFC 2308 says negative caching reduces response time for negative answers and reduces traffic between resolvers and name servers, and it argues negative caching should no longer be seen as optional in a DNS resolver. If you also cache NXDOMAIN or NODATA inside the app, a transient provisioning mistake can linger per process even after the record exists.

There are still narrow cases for a tiny in-process cache: very high query rates, duplicate lookups within a request burst, or libraries that would otherwise resolve repeatedly. Keep it bounded, shorter than the freshness you depend on, and remember that connection pools can pin old backends even after DNS is refreshed.

Why use a node-local caching resolver?

In Kubernetes, a node-local cache improves the outage path more than the happy path. The Kubernetes NodeLocal DNSCache documentation says it runs a DNS caching agent on each node, forwards cache misses upstream, and exposes per-node metrics through CoreDNS running in cache mode. The same documentation calls out practical incident benefits: pods avoid pod-to-CoreDNS DNAT and conntrack overhead, connections from the local caching agent to CoreDNS can use TCP, and this can reduce tail latency tied to dropped UDP packets and DNS timeouts.

The tradeoff is node-level blast radius. A bad cache affects one node’s workloads. In Kubernetes, that is often a better operational boundary than letting every process invent its own cache policy with no shared telemetry. You get one place to inspect hit rate, NXDOMAIN spikes, and upstream timeout behavior.

How does this affect failover in practice?

Worked example: your service connects to db.internal, and failover changes the answer from the old primary to the new one.

With a long application cache, some pods can stay pinned to the dead address until their internal timer expires. With only the Linux/glibc stub, each process can retry nameservers independently, so packet loss or recursive resolver slowness can add retry latency to fresh lookups. In glibc, resolv.conf(5) documents a default timeout of 5 seconds and attempts of 2, while also noting that a single resolver API call does not necessarily map to one timeout. With a node-local cache in Kubernetes, hot names are shared on the node, misses are centralized, and brief upstream problems are less likely to fan out into every pod at once, as described in the NodeLocal DNSCache documentation.

A practical checklist:

• Avoid long app-level DNS caches for movable backends when failover speed matters.
• Use the stub alone when query volume is low and DNS incidents are not a major latency risk.
• In Kubernetes, add a node-local cache when many workloads share names or you need DNS metrics during incidents.
• Test record changes, NXDOMAIN recovery, and resolver timeout behavior separately.
• Measure connection reuse alongside DNS refresh, because DNS alone does not move existing sockets.

Follow-up Q&A?

Should I trust TTLs as exact failover timing?
No. TTLs only help if every cache layer honors them and clients reconnect often enough to observe new answers, consistent with the temporary-cache model described in RFC 1035.

Can a node-local cache hide resolver outages completely?
No. It cushions cache hits and reduces fanout, but cache misses still depend on upstream resolution, as the Kubernetes NodeLocal DNSCache documentation makes clear when it describes forwarding cache misses upstream.

Next step: pick one critical hostname in staging and deliberately test three events separately: record change, NXDOMAIN-to-present recovery, and upstream resolver timeout.

Reviewed: 2026-09-06.