The new Bookworm based images should now be fully "operational" - image cache is refreshed now.
I recommend: * rebase all your PRs - this will make them build much faster using the CI * rebuild your local images for breeze - either when you are asked answer `y` or manually via `breeze ci-image build --python <version>` you can also rebuild images for all python versions in parallel by `breeze ci-image build --run-in-parallel` (fast network recommended) What's remaining is to fix the job that is building "Bullseye" image in CI for backwards compatibility check. Fix for that in https://github.com/apache/airflow/pull/35487 J. On Mon, Nov 6, 2023 at 10:32 AM Jarek Potiuk <ja...@potiuk.com> wrote: > The lazy consensus has been reached. I will proceed with merging the > https://github.com/apache/airflow/pull/35376 > > THanks again Raphael for the reminders and being persistent :) > > On Thu, Nov 2, 2023 at 8:46 PM Jarek Potiuk <ja...@potiuk.com> wrote: > >> Hello everyone, >> >> *TL;DR;* Following our OS upgrade policy [1] - I ask >> for a lazy consensus to switch our Docker images from Bullseye to Bookworm. >> >> The 2.8 version will be based on Bookworm, and we keep an option to build >> a custom Bullseye image for users who need it). In 2.9 we will drop >> Bullseye support from our Dockerfiles. Airflow 2.7 images (if we release >> 2.7.4+) will continue using Bullseye). >> >> Unless someone objects, the lazy consensus will be reached on *Monday >> 6th of November 2023, at midnight CET.* >> >> *More info on the status of PR with change:* >> >> Raphael proposed a PR some time ago and he made our image works for >> Bookworm, and I added a bit of code in Breeze to make it possible to be >> able to select whether Bookworm or Bullseye should be built and the >> resulting joint PR https://github.com/apache/airflow/pull/35376 goes >> through last checks and tests. >> >> Thanks to Raphael for opening the PR and persistently reminding me about >> it and making the Bookworm variant works !!! . It allowed me to implement >> the `--debian-version` switch in almost no time afterwards. >> >> *Explanation about our policies:* >> >> Since the time of buster -> bullseye switch [2] we have had a policy for >> it, so I do not expect much of a discussion here. This PR just implements >> the policy we agreed to - back then. It's a bit earlier than "approximately >> 6 months" as originally described in the policy. It's about 8 months for >> Bullseye to switch to LTS status, but it will likely be ~7 months when we >> release it in 2.8, so I consider it "approximately 6 months". >> >> The bullseye -> bookworm switch should be rather painless for most of our >> users as changes there are minimal and "expected". Also users will still be >> able to take old Dockerfiles and build bullseye images and install even >> newer Airflow versions - even a long time after we stop verifying it in our >> CI (which will happen in 2.9). The buster -> bullseye switch was relatively >> painless, there were a few issues reported for users who used some specific >> versions of some custom software, but the issues were short-lived and went >> away after 1 or 2 months as far as I remember. I expect a very similar >> outcome this time. >> >> *Security implications* >> >> It also turned out, we have another reason to make the migration >> now. This upgrade is very important for security reasons. Bookworm uses >> libssl3 instead of libssl1.1 by default and libsssl1.1 end of life WAS >> September 2023 and switching to it will help to keep Airflow users secure. >> More information here >> https://www.openssl.org/blog/blog/2023/09/11/eol-111/ . Libssl 3 is LTS >> and will be supported until 7th of September 2026. >> >> *References*: >> >> [1] Current policy we have for OS support in images: >> https://github.com/apache/airflow#base-os-support-for-reference-airflow-images >> >> [2] Previous thread for Buster-> Bullseye switch >> https://lists.apache.org/thread/fo20nqb8gs449os1vogjqdd1rv0pxx79 >> >> >> *More context and consequences of the switching (from PR)*. >> >> --------------------- >> >> Debian bookworm (12) is the current stable version of Debian and it is on >> the market for more than a year so all the other dependencies should have >> enough time to catch up. >> >> While Debian bullseye is still supported (oldstable) it will be switching >> to LTS support mode (managed by volunteers) roughly in July 2024 - but we >> want to switch our reference images to bookworm long before that date. >> >> This PR switches our reference images to Debian Bookworm for Dockerfiles >> and images that will be released to Airflow 2.8.0. >> >> Similarly as with the "Debian buster -> Debian bullseye" switch we will >> switch our reference images to bookworm and we will not be publishing >> images based on bullseye. However our users will still be able to build >> custom images using our Dockerfiles with bullseye base image until we >> release Airflow 2.9.0 where the bullseye support will be dropped entirely. >> >> We provide release notes and instructions on how users can build the >> bullseye images if they still want to do it - for example because their >> system level dependencies will require them to do so, but the users are >> advised to switch to bookworm-based images as soon as possible. >> >> The users will likely still be able to build custom Airflow images for >> future airflow releases (using Dockerfiles released with Airflow 2.8), >> however as of Airflow 2.9, we will not release Dockerfiles with support for >> that and we will not verify if Airflow with default dependencies can be >> installed on bullseye Debian. >> >> J. >> >
