Package: aardvark-dns Version: 1.0.3-1+b1 Severity: important DNS resolution of container names seems to fail from inside podman containers with podman 4 and the netavark backend, both with rootless and rootful containers.
>From what I can tell, it looks like aardvark-dns sees the queries, but does not successfully respond, so I'm tentatively filing this against aardvark-dns. Repro steps (rootful) ===================== 1. Verify netavark networking backend is in use: $ sudo podman info | grep networkBackend networkBackend: netavark 2. Create a test network: $ sudo podman network create test-net test-net 3. Create a container attached to the newly created network and attempt to resolve its own name. I've shown two variations of dns lookups here. I believe the first targets aardvark-dns directly, and the second will hit aardvark-dns first, and then fallback to external dns servers. $ sudo podman run -it --rm --name test1 --net test-net alpine / # nslookup test1 10.89.0.1 ;; connection timed out; no servers could be reached / # nslookup test1.dns.podman Server: [host's configured resolver] Address: [host's configured resolver]:53 ** server can't find test1.dns.podman: NXDOMAIN ** server can't find test1.dns.podman: NXDOMAIN Expected results ================ I'd expect test1 and test1.dns.podman to resolve to the container's ip, e.g. 10.89.0.2. Repro steps (rootless) ====================== The repro steps for rootless containers is identical, except without "sudo". $ podman info | grep networkBackend networkBackend: netavark $ podman network create test-net test-net $ podman run -it --rm --name test1 --net test-net alpine / # nslookup test1 10.89.1.1 ;; connection timed out; no servers could be reached / # nslookup test1.dns.podman Server: [host's configured resolver] Address: [host's configured resolver]:53 ** server can't find test1.dns.podman: NXDOMAIN ** server can't find test1.dns.podman: NXDOMAIN Log messages ============ I found these in /var/log/syslog. For the first query (nslookup test1 10.89.0.1) 2022-11-27T19:49:23.967886-08:00 salmon aardvark-dns[0]: Failed while parsing message: unexpected end of input reached 2022-11-27T19:49:23.968293-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message 2022-11-27T19:49:23.975105-08:00 salmon aardvark-dns[0]: Failed while parsing message: unexpected end of input reached 2022-11-27T19:49:23.975589-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message 2022-11-27T19:49:26.478490-08:00 salmon aardvark-dns[0]: Failed while parsing message: unexpected end of input reached 2022-11-27T19:49:26.478981-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message 2022-11-27T19:49:26.479344-08:00 salmon aardvark-dns[0]: Failed while parsing message: unexpected end of input reached 2022-11-27T19:49:26.479612-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message For the second query: 2022-11-27T19:49:49.459055-08:00 salmon aardvark-dns[0]: Failed while parsing message: rdata length too large for remaining bytes, need: 4 remain: 2 2022-11-27T19:49:49.459405-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message 2022-11-27T19:49:49.459732-08:00 salmon aardvark-dns[0]: Failed while parsing message: rdata length too large for remaining bytes, need: 4 remain: 2 2022-11-27T19:49:49.459940-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message 2022-11-27T19:49:51.954116-08:00 salmon aardvark-dns[0]: Failed while parsing message: rdata length too large for remaining bytes, need: 4 remain: 2 2022-11-27T19:49:51.954750-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message 2022-11-27T19:49:51.955148-08:00 salmon aardvark-dns[0]: Failed while parsing message: rdata length too large for remaining bytes, need: 4 remain: 2 2022-11-27T19:49:51.955426-08:00 salmon aardvark-dns[0]: None received while parsing dns message, this is not expected server will ignore this message More info ========= aardvark-dns is running as: /usr/lib/podman/aardvark-dns --config /run/containers/networks/aardvark-dns -p 53 run The aardvark-dns configuration is: $ sudo cat /run/containers/networks/aardvark-dns/test-net 10.89.0.1 e6ffb474399a4defa144fa97a9e889b4b3163ef976874ede7a83aa6814db92d5 10.89.0.2 test1,e6ffb474399a I'm fairly new to podman, and as I understand it, dns resolution is a new feature in podman 4. I *think* it's supposed to work this way, but I could be missing something, and because it's so new, the documentation is hard to find. In case this is a podman issue, I've also included podman version info: podman is 4.3.1+ds1-4 Versions of packages podman depends on: ii conmon 2.1.3+ds1-1 ii crun 1.5+dfsg-1+b1 ii golang-github-containers-common 0.50.1+ds1-2 ii libc6 2.36-5 ii libdevmapper1.02.1 2:1.02.185-2 ii libgpgme11 1.18.0-3 ii libseccomp2 2.5.4-1+b2 ii libsubid4 1:4.13+dfsg1-1 Versions of packages podman recommends: ii buildah 1.28.0+ds1-3 ii catatonit 0.1.7-1+b1 ii dbus-user-session 1.14.4-1 ii fuse-overlayfs 1.9-1 ii slirp4netns 1.2.0-1 ii uidmap 1:4.13+dfsg1-1 Versions of packages podman suggests: pn containers-storage <none> pn docker-compose <none> ii iptables 1.8.8-1 -- System Information: Debian Release: bookworm/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 6.0.0-4-amd64 (SMP w/1 CPU thread; PREEMPT) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled Versions of packages aardvark-dns depends on: ii libc6 2.36-5 ii libgcc-s1 12.2.0-9 aardvark-dns recommends no packages. aardvark-dns suggests no packages. -- no debconf information