DNS, All the Way Down: From getaddrinfo to an Answer

A program has a name and needs an address. Between those two facts sit eight stages, and the first three of them happen entirely inside your own machine, before a DNS query exists at all — in code that predates DNS as a concern and has its own rules, its own configuration files and its own failure modes.

This matters more than it sounds. A large share of the problems people report as “DNS is broken” are not DNS problems, and the tool everyone reaches for first — dig — is structurally incapable of seeing the part of the path where those problems live. It does not read /etc/hosts. It does not load a name service module. It does not sort its results. It cannot tell you what your application saw, because it never asked the question your application asked.

This page walks the full path, in order, from the library call an application makes to the answer coming back and being cached in several places at once. If you want the concepts first, start with the introduction to DNS resolution. This one is the machinery.

Stage 1 — The library call

Applications do not do DNS. They call a C library function and get back a list of socket addresses. On Linux that function is getaddrinfo(3), which the manual describes as combining what gethostbyname(3) and getservbyname(3) used to do, while being reentrant and letting programs stop caring whether the result is IPv4 or IPv6.

Two things about that sentence deserve attention. The first is a list: getaddrinfo returns several addresses, in a deliberate order, and the application almost always tries the first one. The second is the order is not the order DNS gave. Before returning, glibc sorts the addresses according to RFC 6724, which weighs scope, precedence and label to decide whether IPv6 or IPv4 should be tried first.

This is worth dwelling on, because it is a failure that looks exactly like DNS and is not. On a dual-stack host with broken or black-holed IPv6 egress, the AAAA record sorts first because IPv6 has higher precedence in the default table. The application connects to it, and gets a multi-second stall on every single connection before falling back. Meanwhile dig looks perfectly healthy, because dig does not sort and does not connect. The fix is not a DNS change at all:

# /etc/gai.conf
# Promote IPv4-mapped addresses above native IPv6
precedence ::ffff:0:0/96  100

# /etc/gai.conf
# Promote IPv4-mapped addresses above native IPv6
precedence ::ffff:0:0/96  100

Worth knowing while you are in that file: the getaddrinfo(3) manual page still says the sorting is defined in RFC 3484. RFC 3484 was obsoleted by RFC 6724 in 2012. Fourteen years later the primary source is still pointing at the wrong document, which is why so much downstream writing does too. There is also a revision of 6724 sitting in the RFC Editor queue, raising the precedence of unique local addresses above legacy IPv4 and demoting 6to4; it does not have a number yet, so do not let anyone quote you one.

The hints trap

You will read that glibc always applies AI_ADDRCONFIG, which suppresses AAAA lookups on a machine with no IPv6 address configured. That is only true when the caller passes hints as NULL. The manual is explicit: a NULL hints is equivalent to ai_flags being AI_V4MAPPED | AI_ADDRCONFIG. Most real programs pass a non-NULL hints structure with ai_flags set to zero, and in that case AI_ADDRCONFIG is not set and the AAAA query goes out regardless.

That distinction explains a class of “why is this machine sending AAAA queries when it has no IPv6” questions, and it is the reason two programs on the same host can produce different query patterns for the same name.

Current glibc is 2.44, released 25 July 2026. It is worth noting that this code has had a steady drip of memory-safety problems: the 2023 round (CVE-2023-4527, CVE-2023-4806 and CVE-2023-5156) hit forward lookups, with the first only reachable when no-aaaa was set in resolv.conf, AF_UNSPEC was requested, and a response arrived over TCP larger than 2048 bytes. The 2026 round in 2.44 is the mirror image — three fixes in gethostbyaddr and record printing, all reverse lookups and response parsing. Name resolution is one of the few places where untrusted network data is parsed by libc itself, in every process on the machine.

Stage 2 — NSS decides who answers

getaddrinfo does not know what DNS is. It asks the Name Service Switch, and the hosts: line in /etc/nsswitch.conf decides which sources get consulted and in what order. This is where the answer to “why does /etc/hosts work, and when does it not” actually lives.

The systemd project’s own recommended line, from the nss-resolve(8) manual page, is this:

hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns

hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns

Reading the action syntax correctly

A bracketed term is [STATUS=ACTION], and a leading ! negates the status test. There are four statuses a module can report — success, notfound, unavail and tryagain — and by default only success stops the search.

So [!UNAVAIL=return] means: for every status except UNAVAIL, return. If resolve answers at all, whether that answer is an address or a definitive “no such name”, the search stops there. Only if resolved is genuinely not running — the socket is unreachable, the module reports unavail — does the lookup fall through to the rest of the line. That is exactly what makes it safe to put resolve ahead of files.

Compare it with [NOTFOUND=return], which you will see on Avahi’s line as mdns4_minimal [NOTFOUND=return] dns. That one overrides the default continue for a negative answer, so a “not found” from mDNS stops the search dead. The two forms look almost identical and do things that feel opposite. Reading them the wrong way round is one of the more common misunderstandings in this file.

Why files comes after resolve

Putting /etc/hosts second surprises people, and the explanation is worth internalising: on a resolved system, /etc/hosts is normally not read by the files module at all. systemd-resolved reads it itself — ReadEtcHosts= defaults to yes — and answers from it before files is ever reached. The files entry is a fallback for when resolved is down.

Which means that on such a system, “does /etc/hosts work?” is a question about resolved, not about NSS ordering. Since systemd 261 resolved also re-reads /etc/hosts on reload, so a SIGHUP is enough where a restart used to be the folklore answer.

What varies by distribution

Do not trust a table of per-distribution hosts: lines, including one you find here. This is genuinely inconsistent, changes between releases, and is trivially checkable on the machine in front of you:

grep '^hosts:' /etc/nsswitch.conf
readlink -f /etc/resolv.conf
systemctl is-enabled systemd-resolved 2>/dev/null || echo "not installed/enabled"

grep '^hosts:' /etc/nsswitch.conf
readlink -f /etc/resolv.conf
systemctl is-enabled systemd-resolved 2>/dev/null || echo "not installed/enabled"

What can be said with confidence is the shape of the variation. Fedora has enabled systemd-resolved by default since Fedora 33 in 2020, with NetworkManager detecting the stub symlink and turning on split DNS. Arch ships resolved but does not enable it for you. Debian moved systemd-resolved into a separate package and does not use it in a default install. Red Hat’s public networking documentation for RHEL 10 does not mention systemd-resolved anywhere and documents DNS purely as NetworkManager writing /etc/resolv.conf.

There is a second axis, and it is the one people miss: a machine can be using systemd-resolved without having resolve on the hosts: line at all. If /etc/resolv.conf points at 127.0.0.53, then plain nss-dns reaches resolved by sending it a DNS packet. Same daemon, completely different code path in, different behaviour on the edges. Checking one of those two files tells you almost nothing without the other.

And on Alpine, or anything else built on musl, none of this section applies. musl has no NSS at all: no /etc/nsswitch.conf, no modules, no resolve, no myhostname. It reads /etc/hosts, then /etc/resolv.conf, and that is the entire mechanism. Any advice involving module ordering silently does nothing there.

Stage 3 — The stub resolver, and systemd-resolved

A stub resolver is the small piece of code that turns “what is the address of this name” into a DNS query aimed at somebody else’s recursive resolver. glibc has one built in, configured by /etc/resolv.conf. systemd-resolved is a different one, running as a daemon, with its own cache and its own routing rules.

/etc/resolv.conf, and the options that matter

Now a correction that is overdue by nearly a decade. You will still be told, in runbooks and in answers written this year, that long-running processes cache /etc/resolv.conf forever and must be restarted after you edit it. glibc has automatically detected and applied changes to /etc/resolv.conf since version 2.26, released in 2017. The advice was correct once. It has been wrong for nine years and it is repeated constantly.

The search-domain limit has the same shape. glibc capped the search list at six domains and 256 characters up to and including 2.25, and removed the limit in 2.26. musl still ignores a search line longer than 256 characters — not truncates it, ignores it entirely. So the folklore is now true, but of the wrong C library, and in the one environment where search lists get very long.

Two listeners, not one

Everyone knows systemd-resolved listens on 127.0.0.53. Rather fewer know there is a second listener on 127.0.0.54, which the manual describes as operating in proxy mode only: it passes DNS messages upstream relatively unmodified, does not process them locally, does not validate DNSSEC, and offers no LLMNR or multicast DNS.

It exists for software that wants to do its own DNS — its own validation, its own record types, its own cache — while still inheriting resolved’s choice of upstream server and its per-link routing. It is not a raw passthrough; it still decides which upstream your query goes to. If you are running something that insists on speaking DNS itself on a resolved machine, that is the address to point it at.

The three modes of /etc/resolv.conf

• Stub mode — symlink to /run/systemd/resolve/stub-resolv.conf. Lists 127.0.0.53 as the only nameserver, plus the current search domains. This is the recommended mode and what Fedora and Ubuntu ship.
• Static stub mode — symlink to /usr/lib/systemd/resolv.conf. Also 127.0.0.53, but a fixed file with no search domains that never changes. Useful in images.
• Uplink mode — symlink to /run/systemd/resolve/resolv.conf. Lists every upstream server resolved knows about. This bypasses resolved at the packet level: glibc talks straight to the upstream servers and you lose the cache, DNSSEC, DNS over TLS and split DNS.

People switch to uplink mode to “fix” something, it appears to work, and then split-horizon VPN names stop resolving a week later. readlink -f /etc/resolv.conf is the first thing to check on any machine where DNS behaves inexplicably.

What resolved actually defaults to

That first row is where writing about systemd-resolved goes wrong more often than anywhere else. DNSSEC= defaults to no, not to allow-downgrade. The trap is understandable: the same paragraph of the manual page that states the default also recommends setting it to allow-downgrade, and a skim-reader converts the recommendation into the default. Verify it on your own system with resolvectl status rather than believing anybody, including this page.

On encrypted transports: resolved does DNS over TLS and nothing else. There is no DNS over HTTPS and no DNS over QUIC as of systemd 261. This is in motion — a DoH implementation has been open as a pull request since July 2026, proposing configuration like DNS=8.8.8.8#https://dns.google/dns-query{?dns} with no change to defaults — but it is not merged. Both “resolved does DoH” and “resolved will never do DoH” are wrong; “in review” is right. And note that DNSOverTLS=opportunistic buys you very little: the manual page says plainly that it is vulnerable to downgrade, because an on-path attacker simply makes TLS fail and you fall back to cleartext.

Per-link DNS, and the VPN surprise

resolved does not have “a nameserver”. It has a set of servers per network link, plus rules about which names route to which link. A domain written with a leading tilde — ~corp.example — is a routing-only domain: send queries for this suffix to this link’s servers, but do not add it to the search list. A bare corp.example does both. And ~. means “route everything here”, which is how one interface becomes the default DNS route.

The consequence is that resolved will send intranet.corp.example to the VPN’s resolver and www.example.com to the café Wi-Fi’s resolver in the same second. dig cannot reproduce this. Point dig at a specific server and you have destroyed the routing decision that was the entire point. This is why “I tested with dig @1.1.1.1 and got NXDOMAIN, so DNS is broken” is such a persistent false conclusion on VPN-connected laptops.

# Which servers, which domains, which link
resolvectl status

# Which link actually answered, and how
resolvectl query intranet.corp.example --json=pretty

# Watch every query the system makes, live
resolvectl monitor

# Which servers, which domains, which link
resolvectl status

# Which link actually answered, and how
resolvectl query intranet.corp.example --json=pretty

# Watch every query the system makes, live
resolvectl monitor

That last command is the one to add to your habits. resolvectl monitor arrived in systemd 252, and it makes this whole article visible: run it in one terminal, start your application in another, and watch the queries it makes that you never knew about — the search-list expansions, the AAAA lookups, the retries. Two other recent additions are worth knowing for the same reason: resolvectl show-cache (systemd 254) and resolvectl show-server-state (255). None of the three exist in material written before 2023.

A few more recent changes that will not be in older writing. systemd 259 dropped DNS0 from the compiled-in FallbackDNS= list because the service ceased operations, so any config copying that list is stale, and added a Varlink hook interface letting a privileged local service participate in name resolution. systemd 260 moved resolvectl itself from D-Bus to Varlink, which matters if you have scripts poking the D-Bus interface directly. systemd 261 added JSON drop-ins under /etc/systemd/resolve/static.d/ — a generalisation of /etc/hosts that can carry record types beyond A and AAAA — along with DNSCacheSize= and a fix so that answers signed with algorithms resolved does not support are correctly treated as insecure rather than rejected outright.

Stage 4 — The query goes out

Now, finally, DNS. A query is a UDP datagram to port 53 carrying exactly one question: a name, a class and a type. Everything about how many of those datagrams get sent, and how big the answers can be, was decided by configuration in the previous three stages.

The search list, and how one lookup becomes ten

ndots decides whether a name is tried against the search domains before being tried as written. The default is 1: a name containing at least one dot is treated as already qualified. Kubernetes overrides this to 5 in every pod, and still does in current releases.

# A typical pod's /etc/resolv.conf
search myns.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.96.0.10
options ndots:5

# A typical pod's /etc/resolv.conf
search myns.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.96.0.10
options ndots:5

api.github.com has two dots. Two is fewer than five, so it is not treated as qualified, and the stub tries api.github.com.myns.svc.cluster.local, then api.github.com.svc.cluster.local, then api.github.com.cluster.local, and only then api.github.com itself. With glibc sending A and AAAA for each, that is eight queries to resolve one external name, six of which are round trips to the cluster resolver that can only ever return NXDOMAIN.

The mitigations, in the order worth trying them:

1. A trailing dot. https://api.github.com./v3 is fully qualified, so the search list is skipped entirely. No infrastructure change at all. Test it, though — some TLS stacks mishandle the trailing dot in SNI and certificate matching.
2. Per-pod dnsConfig setting ndots to 1 or 2, for workloads that mostly talk to the outside world. This is the mainstream answer. Do not apply it cluster-wide: short internal names like my-service depend on the search list and will stop resolving.
3. NodeLocal DNSCache, which makes the wasted queries cheap instead of eliminating them.
4. CoreDNS autopath, which collapses the sequence server-side. Effective, but it couples the resolver to the client’s search list.

The second option comes with a trap: musl processes the search list differently from glibc, so lowering ndots in an Alpine pod can break resolution that worked fine in a Debian pod on the same cluster. “Just set ndots to 1” is not portable advice across base images.

1232 bytes, and why TCP/53 is now mandatory

EDNS(0) lets a client advertise how large a UDP response it is willing to accept. The historical answer was 4096, and the historical result was IP fragmentation on 1500-byte-MTU paths — fragments that firewalls drop and that are a cache-poisoning vector besides. DNS Flag Day 2020 pushed the industry to 1232.

That is no longer a recommendation; it is a shipped default. BIND changed dig‘s advertised buffer from 4096 to 1232 in version 9.18, and Unbound defaults both edns-buffer-size and max-udp-size to 1232.

The consequence is the part people miss. A smaller UDP ceiling means more responses get truncated, and a truncated response means the client retries over TCP. So TCP port 53 is not an edge case for zone transfers any more — it is ordinary infrastructure that carries a meaningful share of normal lookups, particularly DNSSEC-signed answers, large TXT records and DKIM keys. A firewall rule that permits UDP/53 and drops TCP/53 is a broken configuration in 2026, and it fails intermittently and by record size, which is the hardest possible way to notice.

Which brings up musl again: TCP fallback only arrived in musl 1.2.4, released May 2023, and was settled down in 1.2.5. Alpine 3.18 and earlier cannot follow a truncation to TCP at all, and in a 1232-byte world that means large answers simply fail. Those images are still widely pinned in production, so this is a live bug rather than history.

Two things about what you send

QNAME minimisation (RFC 9156, which obsoleted RFC 7816) stops a recursive resolver from sending the entire name to every server in the chain. Asking the root for www.internal.example.com tells the root operator far more than it needs; under minimisation it is asked only about com. It is on by default in Unbound, in relaxed rather than strict mode, and is widely deployed elsewhere. The visible side effect is that dig +trace output looks different from what older tutorials show.

ANY queries do not mean what they used to. RFC 8482, from 2019, permits a responder to decline a conventional ANY response and return a single record set instead — commonly a synthetic HINFO with the CPU field set to the literal string RFC8482. Implementing it is optional, so behaviour is now inconsistent rather than uniformly changed, which is worse. If you want all the records for a name, enumerate the types. The one honest remaining use of ANY is as a probe of what a particular authoritative server chooses to do.

Stage 5 — A recursive resolver does the work

Your query arrives at a recursive resolver, which has one job: get an answer by any legitimate means, and return it. Most of the time that means reading its own cache and replying in under a millisecond. When it does not, it iterates — asks a root server, gets a referral to the top-level domain servers, asks those, gets a referral to the zone’s authoritative servers, asks those, and gets an answer.

The root of that chain is thirteen server identities, A through M, operated by twelve independent organisations, served from a few thousand anycast instances worldwide — 2004 of them as of 1 September 2026, a number that moves month to month. The thirteen is a property of the letters, not of the hardware; there is no meaningful sense in which there are thirteen machines.

This is also the stage where dig +trace earns its reputation and its caveat. +trace does the iteration from your machine, using your configured resolver only to prime the root list. So it shows you what the public DNS looks like from where you are sitting — which is often not what your configured recursive resolver has cached, or is permitted to see, or has been told to override. When +trace succeeds and ordinary resolution fails, the problem is your resolver, not the zone. That is what the flag is genuinely for, and it is usually left out of the explanation.

The resolver landscape in 2026

Two entries in that table are the same story, and it is the genuinely new thing about DNS software in 2026.

On 11 May 2026, dnsmasq’s maintainer disclosed six CVEs coordinated through CERT, describing them as long-standing bugs applying to essentially all non-ancient versions. They include a heap buffer overflow in name extraction that enables DNS cache injection, an out-of-bounds write in the DHCPv6 code that enables local privilege escalation, an infinite loop in DNSSEC validation, and an information disclosure via client-subnet data. Fixed in 2.92rel2 and 2.93. He also noted explicitly that AI-assisted security research had produced a flood of reports, prompting him to move to faster releases rather than long embargoes.

Separately, ISC has delayed BIND 9.22 to at least the fourth quarter of 2026, because LLM-assisted code analysis surfaced a large number of potential vulnerabilities they are working through. The stable branch therefore remains 9.20, well past its nominal window.

The practical conclusion for dnsmasq is not “stop using it”. It remains the default in libvirt, in NetworkManager’s optional caching mode, and in a very large number of routers and embedded devices, and it is well maintained. The conclusion is that the installed base is the problem: cache injection and local privilege escalation in the component serving your VM network is a different risk posture from 2024, and most consumer routers running it will never receive 2.93.

Stage 6 — The authoritative server answers, or refuses to

At the end of the chain sits a server that holds the zone and answers with authority. It sets the AA bit, it supplies the TTL, and what it says is the ground truth that everything upstream will cache.

The important thing to get right here is that there are three different ways to be told “no”, and conflating them is behind a large fraction of misdiagnoses:

The NODATA case is the one that catches people out on dual-stack machines. It is a perfectly normal, successful response, and any diagnosis that treats it as an error will send you looking in the wrong place.

Negative caching, and why your fix did not take

RFC 2308 defines how long a “no” is remembered, and the rule is more subtle than it is usually quoted. The lifetime of a negative answer is the minimum of the SOA record’s MINIMUM field and the TTL of the SOA record itself — not the MINIMUM field alone. Because it is a minimum, lowering either operand is enough: on a signed lab zone with $TTL left untouched at 3600 and MINIMUM alone dropped to 60, a BIND resolver and an Unbound resolver each cached the “no” for sixty seconds. There is a third term in that minimum too — the resolver’s own negative cap, 3600 in Unbound 1.19.2 and 10800 in BIND 9.18.39 by default — so a MINIMUM of a week is really the resolver’s ceiling in practice. And the SOA’s own TTL reads differently depending on which question you asked: 3600 in the answer section of an SOA query, 60 in the authority section of a negative answer, from the same server in the same second, because the authoritative server applies the first minimum itself before putting the record into the answer. The Life of a DNS Zone has the measurement, and the other four clocks that run alongside this one.

The RFC also observes, in language that has aged well, that values over one day have been found to be problematic and that one to three hours works well.

Here is the operational point. When someone publishes a DNS record, tests it, and reports that the change “has not propagated”, the cause is far more often the negative answer cached during the window when the record did not exist yet than the old positive record. And negative answers are cached independently at every layer — the recursive resolver, systemd-resolved on the client, and sometimes the application. resolvectl flush-caches clears exactly one of those, the one on your own machine.

There is a pleasant twist here that belongs with the next stage. A validating resolver holding signed proof of non-existence can synthesise NXDOMAIN for names it has never asked about — aggressive NSEC caching, on by default in Unbound. So DNSSEC, whose costs are usually all anyone talks about, measurably reduces junk NXDOMAIN traffic to the authoritative servers.

Stage 7 — Validation

DNSSEC proves that the data you received is the data the zone owner published. That is all it does. It does not encrypt anything, it does not hide your queries, and it says nothing about whether the host at the far end is trustworthy. Every record set is signed, each zone’s keys are vouched for by its parent, and the chain terminates at a trust anchor for the root that your resolver carries locally.

An honest account of its deployment has to include the numbers, and the best-sourced ones come from Geoff Huston’s May 2024 analysis for APNIC: roughly 9.4% of the top million domain names are validly signed, about a third of users sit behind a validating resolver, and 3.2% of queries are for signed names — which multiplies out to validation actually happening around 1% of the time, against TLS protecting well over 97% of web requests. Those figures are two years old now and should be quoted with their date attached, but the shape of the argument has not changed: DNSSEC’s costs fall on infrastructure operators while its benefits accrue to domain owners, and applications authenticate independently anyway because they cannot tell whether validation happened.

# Unbound: both keys should be listed
grep -c DNSKEY /var/lib/unbound/root.key
unbound-anchor -l

# systemd-resolved: the anchor comes from the systemd package
resolvectl status | grep -i dnssec

# Unbound: both keys should be listed
grep -c DNSKEY /var/lib/unbound/root.key
unbound-anchor -l

# systemd-resolved: the anchor comes from the systemd package
resolvectl status | grep -i dnssec

Algorithms: the reference moved

If you have ever looked up which DNSSEC algorithms to use, you were sent to RFC 8624. RFC 8624 was obsoleted by RFC 9904 in November 2025. The new document does something structurally interesting: rather than restating the recommendations, it moves the canonical source into the IANA DNSSEC algorithm registries, so guidance can change without anyone having to publish an RFC. Every cheat sheet and answer that says “see RFC 8624” is now pointing at a superseded document.

The current guidance has an asymmetry that “SHA-1 is deprecated” flattens away. Algorithms 5 and 7 (RSA/SHA-1) are MUST NOT for signing, and simultaneously RECOMMENDED for validation, and MUST implement for validation — because signed zones using them still exist and refusing to validate them breaks resolution rather than improving security. Two more that surprise people: algorithm 13 (ECDSA P-256) and algorithm 15 (Ed25519) are the recommended choices for signing, while algorithm 10 (RSA/SHA-512) is not recommended for it.

KeyTrap, and why validators grew budgets

CVE-2023-50387, disclosed February 2024 by researchers at ATHENE, was a flaw in the specification rather than in any one product. DNSSEC requires a validator to try every plausible combination of key and signature before declaring failure. An attacker controlling a zone can craft a response with many colliding keys and signatures, so resolving one query forces an enormous amount of cryptographic work and the resolver goes unresponsive for minutes. CVE-2023-50868 is the sibling attack against NSEC3 closest-encloser proofs.

The interesting outcome is not that a CVE happened. It is that nobody turned DNSSEC off; instead every major validator grew knobs limiting how much work a single answer is permitted to cost. BIND 9.20 carries max-validations-per-fetch and max-validation-failures-per-fetch, both still marked experimental, and equivalent caps went into Unbound, Knot Resolver and PowerDNS Recursor in the same window.

Should you turn validation on?

systemd-resolved does not validate by default. Unbound has the validator in its pipeline by default but only validates once a trust anchor is configured — which distro packages set up for you, so “on by default as packaged” is fair while “on by default from source” is not.

The case for turning it on: it is a checkbox, aggressive NSEC caching gives you a real traffic reduction, and you stop being part of the majority that can be lied to. The case against, stated fairly: about nine in ten of the names you look up are not signed, so you are protecting a tenth of your lookups; the failure mode is hard, in that a remote zone whose operator broke their own signatures becomes SERVFAIL for you and works for everybody else, and you will be the one who has to prove it is not your fault.

What actually breaks when you enable it, in rough order of frequency: zones whose operators let their signatures expire; split-horizon internal zones — an unsigned internal zone under a signed public parent fails validation, and needs a negative trust anchor (resolvectl nta, Unbound’s domain-insecure, BIND’s validate-except); captive portals, which lie about DNS by design; and a stale trust anchor, which is not theoretical this year.

The defensible recommendation for a small server: yes, turn it on — but configure the negative trust anchors for your internal zones first, and make sure whatever ships your trust anchor is being updated.

Stage 8 — Caching, all the way back

The answer now travels back through every layer it came through, and most of them keep a copy under different rules.

• The authoritative server sets a TTL, which is an intention, not an instruction.
• The recursive resolver caps it. BIND, Unbound and the rest all have maximum and minimum cache TTL settings, so a 7-day TTL will often be stored for a day.
• systemd-resolved caches on your machine, positives and negatives, unless Cache=no-negative is set. Inspect it with resolvectl show-cache.
• The application or its runtime may cache in-process. The JVM’s networkaddress.cache.ttl security property is the famous example and has caught out generations of operators; connection pools and HTTP clients in most languages hold resolved addresses for the life of a connection or longer.

None of these layers can flush any of the others. That is the whole content of “DNS propagation”: there is no propagation, only independent expiry, and a record you changed is visible to you and invisible to a colleague on the next desk for entirely legitimate reasons.

The same resolv.conf, two containers, two answers

Containers make every distinction in this article visible at once, because you can run two of them side by side with identical configuration and get different behaviour.

Docker’s two networking cases differ more than most people realise. A container on the default bridge network receives a copy of the host’s /etc/resolv.conf and gets no service discovery by container name. A container on a user-defined network instead gets nameserver 127.0.0.11 — Docker’s embedded resolver, which resolves container names locally and forwards everything else. There is no IPv6 equivalent of that address; the IPv4 one works even in IPv6-only containers.

That copy step is where the classic failure lives. On a host running systemd-resolved, /etc/resolv.conf says nameserver 127.0.0.53 — an address meaningful only in the host’s network namespace. Copied into a container, it points at the container’s own loopback where nothing is listening. Docker handles this by filtering loopback nameservers out, and if nothing is left, substituting public DNS: 8.8.8.8 and 8.8.4.4.

That fallback deserves more attention than it gets. It means a container on a machine you thought was using an internal resolver can silently be sending every query to Google. When Docker’s maintainers tried to extend the internal resolver to the default bridge in 27.0, they reverted it before release — partly because BuildKit, which does not run the internal resolver, then substituted Google’s DNS and broke access to local hostnames, and partly over GDPR and corporate-policy objections to queries leaving quietly. The revert is worth knowing about because it tells you the fallback is real, documented behaviour rather than a bug.

Note also what Docker’s own documentation says about multiple nameservers on the default bridge: the container’s resolver library decides how they are queried, and some libraries query in order while others query in parallel and take the first response even when that response is NXDOMAIN. Docker’s embedded resolver deliberately does the glibc-like thing — in order, stopping on a successful or NXDOMAIN answer.

Which brings us to the Alpine list, all of it a consequence of musl rather than of containers:

• No NSS. No nsswitch.conf, so any advice involving resolve, myhostname or [!UNAVAIL=return] is inapplicable — silently.
• Nameservers are queried in parallel and the first answer wins, including a first answer of NXDOMAIN. With a cluster resolver and a fallback resolver both listed, results become non-deterministic.
• Only three nameservers are honoured.
• A search line over 256 characters is ignored entirely. Kubernetes generates long search lines, so in a deeply namespaced cluster an Alpine pod can lose its whole search list while a Debian pod beside it is fine.
• No TCP fallback before musl 1.2.4 (Alpine 3.19), so pinned older images fail on large answers.
• Different address sorting. musl does not implement the RFC 6724 machinery the way glibc does and has no /etc/gai.conf, so dual-stack preference differs between your Debian and Alpine images.

The summary is worth stating plainly: the same resolv.conf, mounted into two containers, resolves differently — and neither container is doing DNS the way dig does. If you want more on why a container’s view of the system diverges from the host’s, the nine stages between docker run and execve covers the namespaces this all rests on.

The tools, and what each one actually asks

A precision note, because it is easy to get wrong while making the correct larger point: getent hosts and getent ahosts are not the same tool. getent hosts passes the name to gethostbyname2(3) — the obsolete interface. getent ahosts uses getaddrinfo(3) with AF_UNSPEC. Both go through NSS, so either beats dig for this purpose, but only ahosts exercises the dual-stack merge and the sorting that stage 1 was about.

One more rung is worth naming: dig @127.0.0.53. That makes dig talk to the stub rather than around it, which cleanly separates “resolved is misconfigured” from “the upstream is broken”.

nslookup is not deprecated

It is not, and it has not been for over two decades. nslookup is a fully documented, shipped, maintained tool in current BIND 9, sitting in the manual page index alongside dig, host and delv, with its own complete documentation. The deprecation notice everyone remembers was BIND 8-era, from the late 1990s, and it was reversed. The claim has been repeated continuously ever since.

There are real reasons to prefer dig — its output shows detail nslookup omits, nslookup can blur “no answer” and SERVFAIL, and its interactive mode confuses people. Those are the arguments to make. “It’s deprecated” is not one of them, and repeating it makes everything else you say about DNS slightly less trustworthy.

The rest of the toolbox

• delv — uses the same internal resolver and validator logic as named. dig +dnssec shows you records; delv gives you a verdict. It is the right tool for DNSSEC debugging for exactly that reason.
• dig +norecurse — asks a recursive resolver without the RD bit, so it answers only from cache. The way to inspect what a resolver is holding, and to find the one anycast node with a stale entry.
• kdig — from CZ.NIC, packaged as knot-dnsutils. The best dig alternative for encrypted transports: DoT, DoH and DoQ. Maintained by a DNS vendor rather than as a side project.
• doggo — actively maintained, Go, supports DoH, DoT, DoQ and DNSCrypt. Note the naming collision with dog, which is a different program.
• dog — the Rust client that appears in every “modern CLI tools” list. It has had exactly one release, v0.1.0, in November 2020, with dozens of open issues and nothing since. Do not adopt it.

Advice that has expired

DNS folklore is unusually durable, and the reason is structural: these failures are intermittent, the workarounds appear to help, and almost nobody re-tests advice that seemed to work once. Every row below was true when someone first wrote it down.

A worked diagnosis

An application on a laptop cannot reach db.internal.corp.example. The operator has already run dig @1.1.1.1 db.internal.corp.example, received NXDOMAIN, and concluded that DNS is broken. Work the stages instead.

$ getent ahosts db.internal.corp.example
(nothing)

$ dig +short db.internal.corp.example
(nothing)

$ resolvectl status
Link 4 (wg0)
  Current Scopes: DNS
       DNS Servers: 10.20.0.2
        DNS Domain: ~corp.example

$ getent ahosts db.internal.corp.example
(nothing)

$ dig +short db.internal.corp.example
(nothing)

$ resolvectl status
Link 4 (wg0)
  Current Scopes: DNS
       DNS Servers: 10.20.0.2
        DNS Domain: ~corp.example

Both tools fail, so this is not stages 1 to 3 — the split from the box at the top has already ruled out sorting, NSS ordering and /etc/hosts. And resolvectl status shows the routing is configured: ~corp.example routes to the VPN’s resolver on link wg0. So the question is whether the query is reaching that server and what it says.

$ resolvectl query db.internal.corp.example
db.internal.corp.example: resolve call failed: DNSSEC validation failed: no-signature

$ dig @10.20.0.2 +short db.internal.corp.example
10.20.4.17

$ resolvectl query db.internal.corp.example
db.internal.corp.example: resolve call failed: DNSSEC validation failed: no-signature

$ dig @10.20.0.2 +short db.internal.corp.example
10.20.4.17

There it is. The VPN’s resolver has the record and returns it. The stub is throwing the answer away, in stage 7, because corp.example is signed at the public parent while the internal zone underneath it is not — validation fails, and resolved converts that into a hard failure. The correct fix is a negative trust anchor for the internal zone rather than disabling validation wholesale:

$ sudo resolvectl nta internal.corp.example
$ resolvectl query db.internal.corp.example
db.internal.corp.example: 10.20.4.17

$ sudo resolvectl nta internal.corp.example
$ resolvectl query db.internal.corp.example
db.internal.corp.example: 10.20.4.17

Note how badly the original test misled. dig @1.1.1.1 asked a public resolver about a private name and got the only answer it could give. It was not wrong; it was answering a different question, from outside the routing decision that mattered.

What to hold on to

Three things, if nothing else survives the read.

Ask the same question your application asks. getent ahosts first, then resolvectl query, then dig — in that order, narrowing from the whole path down to the protocol. Reaching for dig first skips the three stages where most of the problems are.

Two files determine almost everything about stages 2 and 3, and you have to read both: the hosts: line in /etc/nsswitch.conf, and what /etc/resolv.conf is a symlink to. Either one on its own will mislead you.

Check your trust anchors before October. The root zone changes signing keys this autumn, and the machines that will break are the ones nobody has touched — frozen images, air-gapped resolvers, appliances. That is a job with a deadline on it, unlike most of what is on this page.

For the layer underneath all of this — what actually happens to the UDP datagram once the stub hands it to the kernel — see the nine stages a packet passes through between a socket and the wire. And for the most common real-world instance of the TCP-port-53 problem above, see The Life of an Outbound Email: a 4096-bit DKIM key answers in 834 bytes, a resolver that cannot fall back to TCP gets a truncated answer, and a correctly signed message fails authentication at that receiver and nowhere else.