DNS redirect failure is only a warning: enforce mode runs with DNS capture silently off when `iptables` is absent

[matthewdevenny]

Symptom

On a self-hosted ARC runner (Kubernetes, enforce mode, cargowall v1.3.6), SetupDNSRedirect failed because the runner image has no iptables. The run continued into enforce mode with the redirect never installed:

08:57:10.123 Docker DNS interception enabled docker_bridge=172.17.0.1
::warning::Failed to configure Docker DNS error=failed to write daemon.json: open /etc/docker/daemon.json: no such file or directory
::warning::Failed to set up DNS redirect (iptables) error=iptables -A OUTPUT [...] failed: exec: "iptables": executable file not found in $PATH
...
08:57:15.012 CargoWall TC firewall started interface=eth0 ... default_action=deny

Eleven seconds later, connections to Azure blob storage began failing, and kept failing for the rest of the run:

08:57:26.410 Connection blocked src=...:50982 dst=20.209.108.193 dst_ip=20.209.108.193 dst_port=443 process= pid=0
08:57:27.436 Connection blocked src=...:50982 dst=20.209.108.193 ...      (7 SYN retries, new source port, repeat)

Note dst= carries the raw IP, not a hostname. Compare a working line from the same run:

08:57:20.316 Connection allowed ... dst=weureplstore225.blob.core.windows.net dst_ip=57.150.167.1 dst_port=443

The policy allowed blob.core.windows.net, *.blob.core.windows.net and **.core.windows.net on 443 the whole time. The blocked IPs appear in no DNS resolution: added to firewall line — no DNS answer ever mapped them, because the queries never reached the proxy.

Why the run kept going

SetupDNSRedirect failure is a logger.Warn (cmd/start.go:347-350):

if err := network.SetupDNSRedirect(logger); err != nil {
    logger.Warn("Failed to set up DNS redirect (iptables)", "error", err)
} else {
    dnsRedirectActive = true
}

dnsRedirectActive stays false, and the only thing it gates downstream is the systemd-resolved cache flush (cmd/start.go:760). Nothing else notices; the TC program attaches and enforces normally.

The iptables DNAT is the only mechanism that forces an arbitrary resolver through the proxy. Without it, capture depends on every client voluntarily using the proxy address — and in this run two resolver paths were both reachable and invisible:

• 168.63.129.16:53/udp, auto-allowed as azure_infrastructure
• 8.8.8.8 / 8.8.4.4:53, from the policy's own dns.google rule

So the environment fails into the worst shape available: DNS resolves fine, enforcement is fully armed, and the resulting IPs are unattributed and denied. It presents to the operator as an intermittent, destination-specific policy gap rather than as a broken capture path.

Why it looks intermittent

In this run some weureplstore*.blob.core.windows.net names did go through the proxy and were allowed; others never did and were blocked on the raw IP. Same class of destination, different resolver. The operator's natural response is to add the blocked IP as a /32 — and this policy already carried ~20 such /32s across 20.209.0.0/16 and 57.150.0.0/16, i.e. previous rounds of the same failure against rotating Azure storage stamps. The DNS-proxy invariant says a block on an unattributed IP means a capture bypass; warning-and-continuing here trains operators to do the opposite.

Environment (verified)

ghcr.io/actions/actions-runner does not carry iptables. Verified against 2.337.0 (sha256:e5496277be5d09bc968b3d64911b74e219ac4a3f2edce956a3ecf9271bea1ef4) and latest (built 2026-08-27):

== command -v iptables ==     NOT FOUND on PATH
== find / -name "iptables*" ==  (nothing)
== dpkg -l | grep -iE "iptables|nftables" ==  no iptables/nftables package installed

The image's complete apt set is sudo lsb-release gpg-agent software-properties-common curl jq unzip plus git (Ubuntu 24.04 noble base). nft, conntrack and ip (iproute2) are absent too; docker (CLI, from the static tarball) and systemctl are present.

This is the stock ARC runner image, so it is not an exotic configuration — it is the default for every ARC user.

Proposal

1. Preflight exec.LookPath("iptables") when DNSRedirectIptables is set and fail with a message naming the package. The github-action and gitlab-ci presets turn the flag on (cmd/flags.go:333), so this is the common path, not an opt-in. Erroring at startup costs seconds; a silent capture gap costs a full debugging cycle and leaves a policy polluted with CDN /32s.
2. If a non-fatal path is wanted instead, it has to be loud in the run summary — a single ::warning:: among others is demonstrably not enough.

Secondary: the two Docker warnings around it

In a container-hosted runner the daemon is not locally manageable, so ConfigureDockerDNS and the daemon restart both fail by construction:

::warning::Failed to configure Docker DNS error=... open /etc/docker/daemon.json: no such file or directory
::warning::Failed to restart Docker daemon for DNS config error=systemctl restart docker failed: exit status 1

Both are expected in this topology and neither needs fixing — but they bracket the real error and read as equally severe, so the log presents as three independent problems when only one mattered.

GetDockerBridgeIP succeeding (it reads the real docker0, pkg/network/docker.go:64) while /etc/docker is absent is a reliable signal that a Docker daemon shares the network namespace but not the filesystem — a dind sidecar or a mounted node socket. In that case the daemon.json path can be skipped with one explanatory line, because the redirect covers the same ground: nat OUTPUT rules installed from this container apply to the whole shared netns, and the DNAT target 127.0.0.1:53 is the shared loopback.