Hello everyone, I am just about to complete preparation to make our images (and CI) work on Debian 11 images (Bullseye) vs Debian 10 (Buster). All looks (almost) good, but I would like to tap into the collective wisdom of the group here to decide on next steps (and maybe propose a policy we might be following in the future). The last change that allows to easily switch between the two is in reviee: https://github.com/apache/airflow/pull/21546
Context/where we are: For quite a while we used Buster (Debian 10) as our base image. Buster is a stable release and it is being replaced with the next stable release Bullseye (Debian 11). The Release schedule is here: https://wiki.debian.org/DebianReleases The most important facts from those: * Bullseye was introduced in August 2021 (6 months ago) and has no end-of-life yet * Buster was introduced in July 2019 and its end of life is ~ August 2020 (approx. 6 months from now) Python default images are (as of a few months), However they provide (as usual -buster and -bullseye) specific images. The Bullseye switch is really needed to support ARM (M1) images because of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989604 (still unresolved on Buster). It seems appropriate to switch to Bullseye now, however there is one problem with it - MSSQL drivers do not yet support Bullseye. I've reached out to the maintainers and await their answer when it might be available: https://github.com/MicrosoftDocs/sql-docs/issues/7255#issuecomment-1037097131 . This is the last blocker that prevents full switch (failing PR for a full switch is https://github.com/apache/airflow/pull/21378). All tests pass except MSSQL. But I think we need to proceed, regardless of the timeline and switch to Bullseye. Question: We need to decide on what support we give our users in the images now, in the 2.3.0 release and in the future. We have no policies for that yet, so it might be a good time to decide now. Some of the users might "rely" on the Buster being used by the images as they might have some incompatible libraries (similarly to MSSQL). And I believe we need to support Buster till at least the end of life of it. This support might be twofold: * we can publish in https://hub.docker.com/r/apache/airflow our binary images with Buster and/or Bullseye * we can only publish one of those (and change from Buster to Bullseye for 2.3.0) but users will be able to build their own custom images using our Dockerfile for either of those. Proposal: I think there are multiple reasons why we should support both Bullseye and Buster for the coming 6 months. We might build both images in CI and add a new "matrix" dimension and run most tests on Bullseye but some tests on Buster (specifically MSSQL until Bullseye is supported). We can publish both types of images (with -buster, -bullseye suffixes same as Python) but the default image should be changed to Bullseye. This will add some complexity (and build overhead) to our CI, but I think it's worth it. Then we could drop support and release the -buster images after it reaches end of life (but we will leave the users possibility of building their own buster images without guarantee that it will work). I think this might become our Policy also for the future (6 months before end-of-life we switch by default to the new stable and provide built images until end-of-life). WDYT? Does it look like a good proposal? J.
