7. The wire protocol

DNS on the wire is one of the simplest protocols still in production, and the oldest constraint in it still decides how zones are designed.

   THE MESSAGE, 12-BYTE HEADER THEN FOUR
   SECTIONS

   ┌─────────────────────────────────────┐
   │ ID            16 bits               │
   ├─────────────────────────────────────┤
   │ QR OPCODE AA TC RD RA Z AD CD RCODE │
   ├─────────────────────────────────────┤
   │ QDCOUNT  — questions                │
   │ ANCOUNT  — answers                  │
   │ NSCOUNT  — authority records        │
   │ ARCOUNT  — additional records       │
   ├─────────────────────────────────────┤
   │ QUESTION    name, type, class       │
   │ ANSWER      the records asked for   │
   │ AUTHORITY   NS records, or an SOA   │
   │ ADDITIONAL  glue, OPT, hints        │
   └─────────────────────────────────────┘

The flags, each of which you will meet in dig

   QR   0 = query, 1 = response
   AA   Authoritative Answer. the server
        owns this zone.
   TC   TRUNCATED. "my answer did not fit;
        ask again over TCP."
   RD   Recursion Desired. the client is
        asking the server to do the work.
   RA   Recursion Available. the server is
        willing to.
   AD   Authentic Data. DNSSEC validated.
        (chapter 9)
   CD   Checking Disabled. "do not validate
        for me."

dig example.com | head -8
dig +norecurse @a.root-servers.net example.com | grep flags
dig +dnssec example.com | grep flags

   READING FLAGS IS THE FASTEST DIAGNOSTIC
   IN DNS

   `aa` present   → you reached an
                    authoritative server
   `aa` absent    → a resolver, or a
                    referral
   `tc` present   → truncation; UDP size
                    problem
   `ra` absent on
     a resolver   → it refuses to recurse
                    for you
   `ad` present   → DNSSEC validated

Response codes

   0  NOERROR    success (possibly with
                 zero answers — NODATA)
   1  FORMERR    malformed query
   2  SERVFAIL   the server broke, OR
                 DNSSEC validation failed.
                 THESE ARE THE SAME CODE,
                 which is why DNSSEC
                 failures are so hard to
                 diagnose.
   3  NXDOMAIN   the name does not exist
   4  NOTIMP     opcode not implemented
   5  REFUSED    policy: not my zone, or
                 you are not allowed
   9  NOTAUTH    not authoritative (dynamic
                 update)
   16 BADVERS/
      BADSIG     EDNS version or TSIG issue

Caution

SERVFAIL meaning both “the server has a problem” and “DNSSEC said no” is the single worst piece of protocol design in daily DNS operations. A domain that is unresolvable with SERVFAIL from validating resolvers and resolves fine with +cd is a DNSSEC failure, not a server failure — and that one-flag test is the fastest way to tell them apart.

The 512-byte limit

   RFC 1035 (1987): A DNS MESSAGE OVER UDP
   MUST NOT EXCEED 512 OCTETS.

   WHY: 576 bytes was the guaranteed
   minimum IPv4 reassembly buffer, minus
   headers. it was a promise that any
   host could receive the reply.

   CONSEQUENCES THAT ARE STILL WITH US
     − 13 root server names
     − `ANY` queries as amplifiers
     − zones designed around answer size
     − truncation and TCP fallback
     − DNSSEC being hard to deploy

dig +bufsize=512 +ignore example.com DNSKEY | grep -E 'flags|MSG SIZE'
dig +notcp +bufsize=512 . NS

EDNS0, and the 1232-byte compromise

   RFC 6891: PUT A PSEUDO-RECORD IN THE
   ADDITIONAL SECTION, AND USE ITS FIELDS
   AS PROTOCOL EXTENSIONS.

   OPT RR
     name:  "." (root)
     type:  41
     CLASS field  → reused as the
                    advertised UDP payload
                    size
     TTL field    → reused as extended
                    RCODE + version + flags
                    (DO bit lives here)
     RDATA        → options: NSID, ECS,
                    cookies, padding,
                    extended errors

   WHAT SIZE TO ADVERTISE

   4096   the long-standing default. and
          the cause of IP FRAGMENTATION,
          which middleboxes drop, which
          produces intermittent failures
          that look like nothing.

   1232   the "DNS Flag Day 2020"
          recommendation: 1280 (the IPv6
          minimum MTU) minus 48 bytes of
          headers. chosen so a DNS reply
          NEVER fragments.

   1400 / 1480  seen in the wild, tuned to
          Ethernet MTU.

dig +edns=0 +bufsize=1232 example.com
dig +nsid @k.root-servers.net . SOA | grep NSID
dig +subnet=203.0.113.0/24 example.com     # EDNS Client Subnet

Advertising 1232 is now the default in Unbound, BIND, Knot and the major public resolvers. The reason it matters is subtle: a fragmented UDP reply is also a security problem, because an attacker who can inject a second fragment can rewrite part of a response without guessing the ID at all.

Extended DNS Errors

   RFC 8914 FINALLY SPLITS SERVFAIL APART.

   dig returns, in the OPT RR:
     EDE 6   DNSSEC Bogus
     EDE 7   Signature Expired
     EDE 9   DNSKEY Missing
     EDE 10  RRSIGs Missing
     EDE 18  Prohibited (policy/blocking)
     EDE 22  No Reachable Authority

   SUPPORTED BY: Unbound, BIND 9.16+,
   Knot, 1.1.1.1, 9.9.9.9. check for it
   before guessing.

dig +dnssec dnssec-failed.org | grep -i 'EDE'
kdig +dnssec dnssec-failed.org | grep -i 'extended'

Truncation and TCP

   IF THE ANSWER DOES NOT FIT IN THE
   ADVERTISED SIZE

   1. the server sends what fits, with the
      TC flag set
   2. the client RE-ASKS over TCP, port 53
   3. over TCP, messages are length-
      prefixed with two octets, and there
      is no size limit

   DNS OVER TCP IS NOT A FALLBACK HACK.
   RFC 7766 made it a REQUIREMENT. a
   firewall blocking TCP/53 breaks DNSSEC,
   large zones and zone transfers.

dig +tcp example.com
dig +notcp +ignore example.com DNSKEY | grep 'flags'
ss -tlnp | grep ':53'
nc -zv ns1.example.com 53

Note

“We only allow UDP/53 through the firewall” is a configuration that worked in 2005 and breaks DNSSEC today. TCP/53 must be open in both directions for any resolver, and it is a standing item worth checking whenever a domain works from one network and not another.

Name compression

   NAMES REPEAT CONSTANTLY IN A MESSAGE, SO
   A LABEL MAY BE A POINTER.

   A LENGTH BYTE WITH ITS TOP TWO BITS SET
   (0b11xxxxxx) MEANS "the rest of this
   name is at offset X in this message".

   ;; a reply about www.example.com
   QUESTION:  3 w w w 7 e x a m p l e 3 c o m 0
                      ↑ offset 4
   ANSWER:    0xC0 0x04     ← "see offset 4"

Compression is why a reply containing twenty records about one zone is far smaller than twenty times one record, and why answer-size estimates based on text representation are always too large. It is also a historical source of parser bugs: a malicious pointer loop was a denial-of-service vector in several implementations, and every parser now has a loop guard.

DNS cookies, in a little more detail

Cookies deserve more than a line in the defence list, because they solve the spoofing problem without the cost of TCP.

   RFC 7873 / 9018

   CLIENT COOKIE  8 bytes, derived from the
                  client IP, the server IP
                  and a client secret
   SERVER COOKIE  8–32 bytes, computed by
                  the server over the client
                  cookie, the client IP and
                  a server secret

   FIRST QUERY   client sends its cookie;
                 server replies with both
   LATER QUERIES client echoes the server
                 cookie, which PROVES it
                 received a previous reply
                 — therefore its source
                 address is real

dig +cookie example.com | grep -i cookie
dig +cookie=0102030405060708 example.com

   WHAT IT BUYS
     a server can rate-limit or refuse
     cookieless clients without blocking
     anybody legitimate, because real
     resolvers get a cookie on their first
     exchange.

   WHAT IT COSTS
     essentially nothing: 8–40 bytes in an
     EDNS option, no round trips, no state
     on the server (the cookie is a MAC,
     not a table entry).

Padding, and why it is in the same place

RFC 7830’s padding option also lives in the OPT record, and exists because encrypted DNS (chapter 10) hides the content of a query but not its length — and query lengths are distinctive enough to fingerprint which domain was asked for. Padding to a block size removes that channel, at the cost of bandwidth, and is standard practice in DoT and DoH deployments.

Amplification, and the reflection problem

   THE ATTACK

   1. a small query (60 bytes) with a
      SPOOFED source address
   2. sent to an OPEN RESOLVER or an
      authoritative server
   3. producing a large reply (4000 bytes)
   4. delivered to the victim

   AMPLIFICATION FACTOR: 50× to 180×
   with the right query (ANY, DNSKEY, a
   large TXT).

   THE DEFENCES, IN LAYERS

   DO NOT RUN AN OPEN RESOLVER
     the single biggest one. restrict
     recursion to your own networks.

   RESPONSE RATE LIMITING (RRL)
     authoritative servers drop repeated
     identical responses to the same
     prefix. standard in BIND, Knot, NSD.

   REFUSE ANY
     RFC 8482: answer ANY with a minimal
     synthetic reply or HINFO.

   DNS COOKIES (RFC 7873)
     a lightweight shared secret in an
     EDNS option. proves the client's
     address is real, without TCP.

   BCP 38
     networks not forwarding spoofed
     source addresses. the actual fix,
     and the least deployed.

# am I an open resolver? (ask from outside your network)
dig +short @your.public.ip example.com
# RRL in BIND
grep -A5 rate-limit /etc/bind/named.conf.options
# cookies
dig +cookie example.com | grep COOKIE

The amplification problem is why so much of this chapter’s modern design — cookies, ANY refusal, RRL, the 1232-byte size — exists. A stateless single-packet protocol that returns more than it receives is a weapon unless carefully constrained, and almost every protocol extension since 2010 has that constraint in mind.

What to take from this chapter

• A DNS message is a 12-byte header plus four sections: question, answer, authority, additional.
• Reading flags is the fastest diagnostic available: aa means you reached authority, tc means truncation, missing ra means the server will not recurse for you, ad means DNSSEC validated.
• SERVFAIL means both “server broke” and “DNSSEC said no”, which is the worst piece of protocol design in daily operations. dig +cd distinguishes them in one command.
• The 512-byte UDP limit from 1987 still shapes everything: 13 root names, ANY as an amplifier, zone design, and DNSSEC’s deployment difficulty.
• EDNS0 adds an OPT pseudo-record and reuses its class and TTL fields for a payload size and extended flags, which is where the DNSSEC DO bit, NSID, Client Subnet, cookies and padding all live.
• Advertise 1232 bytes, not 4096. Fragmented UDP replies are dropped by middleboxes and let an attacker rewrite part of a response without guessing the ID.
• Extended DNS Errors (RFC 8914) finally split SERVFAIL into specific causes. Check for them before guessing.
• DNS over TCP is a requirement, not a fallback (RFC 7766). Blocking TCP/53 breaks DNSSEC, large zones and transfers.
• Name compression replaces repeated names with pointers, which is why answers are far smaller than their text form suggests — and why every parser needs a pointer-loop guard.
• Amplification is why modern DNS looks the way it does: do not run an open resolver, enable RRL, refuse ANY, use cookies, and BCP 38 remains the real fix nobody deploys.

Next: zones and delegation — the SOA fields, zone transfers, NOTIFY, dynamic updates, and keeping parent and child in step.