potiuk opened a new pull request, #76200: URL: https://github.com/apache/airflow/pull/76200
The CI image build on `main` fails while installing OS dependencies on the hardened images, for example the [arm64 / Python 3.13 build](https://github.com/apache/airflow/actions/runs/41285189396/job/124785242869): ``` gcc-14 : Depends: gcc-14-base (= 14.2.0-19+deb13u0+dhi5) but 14.2.0-19+dhi5 is to be installed libgmp-dev : Depends: libgmp10 (= 2:6.3.0+dfsg-3+deb13u0+dhi2) but 2:6.3.0+dfsg-3+dhi2 is to be installed ``` Docker renumbered its builds again (`14.2.0-19+dhi5` became `14.2.0-19+deb13u0+dhi5`, which sorts lower) and the dhi.io apt repository dropped the old numbers, while the base images pinned in #76023 still ship them (docker-hardened-images/catalog#689). At the dhi.io pin priority of 999, apt will not downgrade the image's copies, so everything depending on the exact new versions (gcc, libgomp1 via graphviz, libgmp-dev) becomes uninstallable. Pinning the base image to a digest cannot prevent this: the image is frozen, but the apt repository it installs from is live. The install script now moves only the installed packages whose version no repository offers any more to dhi.io's current build, before installing anything else. Packages still available are left alone, so PGDG's newer libpq5 keeps resolving. Tested in the pinned arm64 Python 3.13 base image: 10 renumbered packages (`gcc-14-base`, `libgcc-s1`, `libstdc++6`, `libgmp10`, `libzstd1`, `tar`, ...) were aligned, after which `build-essential libgmp-dev graphviz libgomp1` resolved cleanly. shellcheck reports no findings. ### Follow-up This unblocks `main`, but CI builds still install system packages from a live repository, so the same commit can build a different image on a different day. A follow-up PR will pin the OS-dependency layer itself: a scheduled job builds the base image plus `install_os_dependencies.sh` and pushes it to GHCR by digest, the CI image starts from that digest, and the existing upgrade check moves the pin only after the full CI has passed. Day-to-day CI then no longer talks to dhi.io for system packages, and a dhi.io change can only break the upgrade PR, not `main`. Standalone builds of the `Dockerfile` with a custom base image keep installing live, protected by the change in this PR. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Opus 5 Generated-by: Claude Opus 5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
