GitHub user will-m-buchanan created a discussion: Can't build docker images based on slim-2.8.0 or higher
Our Airflow deployment is woefully behind the times and we're finally getting around to upgrading. First step before v3 is to upgrade from 2.7.3 to 2.11.2. We currently use the base image `apache/airflow:slim-2.7.3-python3.10` and run some `apt-get` commands to install other necessary utils. Unfortunately, when trying to build the image in Gitlab with a docker-in-docker runner, the 2.11.2 version of the base image (or any version 2.8.0 and above) breaks with the following error during `apt-get update`: ``` #11 127.9 Get:1 http://deb.debian.org/debian bookworm InRelease [151 kB] #11 127.9 Get:2 http://deb.debian.org/debian bookworm-updates InRelease [55.4 kB] #11 127.9 Get:3 http://deb.debian.org/debian-security bookworm-security InRelease [48.0 kB] #11 127.9 Get:4 https://packages.microsoft.com/debian/12/prod bookworm InRelease [3618 B] #11 127.9 Err:1 http://deb.debian.org/debian bookworm InRelease #11 127.9 The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 78DBA3BC47EF2265 NO_PUBKEY F8D2585B8783D481 #11 128.0 Err:2 http://deb.debian.org/debian bookworm-updates InRelease #11 128.0 The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 78DBA3BC47EF2265 #11 128.0 Err:3 http://deb.debian.org/debian-security bookworm-security InRelease #11 128.0 The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54404762BBB6E853 NO_PUBKEY BDE6D2B9216EC7A8 #11 128.0 Err:4 https://packages.microsoft.com/debian/12/prod bookworm InRelease #11 128.0 At least one invalid signature was encountered. #11 128.1 Get:5 https://download.docker.com/linux/debian bookworm InRelease [46.6 kB] #11 128.1 Err:5 https://download.docker.com/linux/debian bookworm InRelease #11 128.1 The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8 #11 128.1 Get:6 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease [180 kB] #11 128.2 Err:6 https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease #11 128.2 At least one invalid signature was encountered. #11 157.9 Ign:7 https://archive.mariadb.org/mariadb-10.11/repo/debian bookworm InRelease #11 188.9 Ign:7 https://archive.mariadb.org/mariadb-10.11/repo/debian bookworm InRelease #11 221.0 Ign:7 https://archive.mariadb.org/mariadb-10.11/repo/debian bookworm InRelease #11 255.0 Err:7 https://archive.mariadb.org/mariadb-10.11/repo/debian bookworm InRelease #11 255.0 Could not wait for server fd - select (11: Resource temporarily unavailable) [IP: 138.201.152.105 443] #11 255.0 Reading package lists... #11 255.0 W: GPG error: http://deb.debian.org/debian bookworm InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 78DBA3BC47EF2265 NO_PUBKEY F8D2585B8783D481 #11 255.0 E: The repository 'http://deb.debian.org/debian bookworm InRelease' is not signed. #11 255.0 W: GPG error: http://deb.debian.org/debian bookworm-updates InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 6ED0E7B82643E131 NO_PUBKEY 78DBA3BC47EF2265 #11 255.0 E: The repository 'http://deb.debian.org/debian bookworm-updates InRelease' is not signed. #11 255.0 W: GPG error: http://deb.debian.org/debian-security bookworm-security InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 54404762BBB6E853 NO_PUBKEY BDE6D2B9216EC7A8 #11 255.0 E: The repository 'http://deb.debian.org/debian-security bookworm-security InRelease' is not signed. #11 255.0 W: GPG error: https://packages.microsoft.com/debian/12/prod bookworm InRelease: At least one invalid signature was encountered. #11 255.0 E: The repository 'https://packages.microsoft.com/debian/12/prod bookworm InRelease' is not signed. #11 255.0 W: GPG error: https://download.docker.com/linux/debian bookworm InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 7EA0A9C3F273FCD8 #11 255.0 E: The repository 'https://download.docker.com/linux/debian bookworm InRelease' is not signed. #11 255.0 W: GPG error: https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease: At least one invalid signature was encountered. #11 255.0 E: The repository 'https://apt.postgresql.org/pub/repos/apt bookworm-pgdg InRelease' is not signed. #11 255.0 E: Problem executing scripts APT::Update::Post-Invoke 'rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true' #11 255.0 E: Sub-process returned an error code ``` The build runs successfully on my local machine (running docker v28.1.1), but only fails in Gitlab. The Gitlab CI/CD job is defined with ```yaml image: docker:28.5.2-cli services: - name: docker:28.5.2-dind ``` I've seen many answers to similar issues that suggest something along the lines of including ``` sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys <list of missing keys> ``` in the Dockerfile, but this fails with "Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8))." (supposedly just a warning but it fails without any other message) I've also tried ``` apt-get install -y --no-install-recommends debian-archive-keyring ``` But that just tells me "debian-archive-keyring is already the newest version (2023.3+deb12u2). debian-archive-keyring set to manually installed." then continues to fail as normal. Finally, I attempted to just run `curl -v http://deb.debian.org/debian/dists/bookworm/InRelease` to see if I could even download one of the troublesome files directly and got this error ``` #5 [2/3] RUN curl -v https://deb.debian.org/debian/dists/bookworm/InRelease #5 0.365 * getaddrinfo() thread failed to start #5 0.365 * Could not resolve host: deb.debian.org #5 0.365 * Closing connection 0 #5 0.365 curl: (6) getaddrinfo() thread failed to start ``` So it seems maybe there's some kind of issue with DNS inside the docker-in-docker container. What I'm curious about is what changed between 2.7.3 and 2.8.0 so that earlier images were not affected by this DNS issue? Also, if anybody has any suggestions I would be very grateful. GitHub link: https://github.com/apache/airflow/discussions/64583 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
