Yeah. That would definitely be a simpler approach for us. What we can also do is we can still make sure that you can actually "build" Buster images if you want and we could run the simple tests we already have for those images (those tests just check very basic stuff - like if expected packages are importable, if Airflow commands work, if you can run airflow as root user etc.. This would be very little to no complication of the CI and we could provide a recipe for the users how to build their own buster image - but without guarantees that everything works there.
This way might provide the users with a "workable solution" while we could keep it simple for us and put a strong emphasis on "If you want to get full support - also upgrade to the latest stable OS". J. On Tue, Feb 15, 2022 at 9:53 AM Ash Berlin-Taylor <[email protected]> wrote: > > My vote is to just move forward and only release Bullseye images for 2.3 > onwards > > On 15 February 2022 08:43:50 GMT, Jarek Potiuk <[email protected]> wrote: >> >> Good news from today - I got the response from Microsoft and it seems >> that MSSQL Bullseye support is going to be released today :) . So at >> least that problem is going to be solved. Still the question of >> whether we should build/release both Bullseye and Buster ? >> >> J. >> >> On Mon, Feb 14, 2022 at 2:14 PM Jarek Potiuk <[email protected]> wrote: >>> >>> >>> And Python default images are Bullseye now. >>> >>> On Mon, Feb 14, 2022 at 2:13 PM Jarek Potiuk <[email protected]> wrote: >>>> >>>> >>>> Buster end of life is August 2022 of course :) >>>> >>>> On Mon, Feb 14, 2022 at 2:12 PM Jarek Potiuk <[email protected]> wrote: >>>>> >>>>> >>>>> Hello everyone, >>>>> >>>>> I am just about to complete preparation to make our images (and CI) work >>>>> on Debian 11 images (Bullseye) vs Debian 10 (Buster). All looks (almost) >>>>> good, but I would like to tap into the collective wisdom of the group >>>>> here to decide on next steps (and maybe propose a policy we might be >>>>> following in the future). The last change that allows to easily switch >>>>> between the two is in reviee: https://github.com/apache/airflow/pull/21546 >>>>> >>>>> Context/where we are: >>>>> >>>>> For quite a while we used Buster (Debian 10) as our base image. Buster >>>>> is a stable release and it is being replaced with the next stable release >>>>> Bullseye (Debian 11). >>>>> >>>>> The Release schedule is here: https://wiki.debian.org/DebianReleases >>>>> >>>>> The most important facts from those: >>>>> * Bullseye was introduced in August 2021 (6 months ago) and has no >>>>> end-of-life yet >>>>> * Buster was introduced in July 2019 and its end of life is ~ August >>>>> 2020 (approx. 6 months from now) >>>>> >>>>> Python default images are (as of a few months), However they provide (as >>>>> usual -buster and -bullseye) specific images. >>>>> >>>>> The Bullseye switch is really needed to support ARM (M1) images because >>>>> of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989604 (still >>>>> unresolved on Buster). >>>>> >>>>> It seems appropriate to switch to Bullseye now, however there is one >>>>> problem with it - MSSQL drivers do not yet support Bullseye. I've reached >>>>> out to the maintainers and await their answer when it might be available: >>>>> https://github.com/MicrosoftDocs/sql-docs/issues/7255#issuecomment-1037097131 >>>>> . This is the last blocker that prevents full switch (failing PR for a >>>>> full switch is https://github.com/apache/airflow/pull/21378). All tests >>>>> pass except MSSQL. >>>>> >>>>> But I think we need to proceed, regardless of the timeline and switch to >>>>> Bullseye. >>>>> >>>>> Question: >>>>> >>>>> We need to decide on what support we give our users in the images now, >>>>> in the 2.3.0 release and in the future. We have no policies for that yet, >>>>> so it might be a good time to decide now. Some of the users might "rely" >>>>> on the Buster being used by the images as they might have some >>>>> incompatible libraries (similarly to MSSQL). And I believe we need to >>>>> support Buster till at least the end of life of it. >>>>> >>>>> This support might be twofold: >>>>> >>>>> * we can publish in https://hub.docker.com/r/apache/airflow our binary >>>>> images with Buster and/or Bullseye >>>>> * we can only publish one of those (and change from Buster to Bullseye >>>>> for 2.3.0) but users will be able to build their own custom images using >>>>> our Dockerfile for either of those. >>>>> >>>>> Proposal: >>>>> >>>>> I think there are multiple reasons why we should support both Bullseye >>>>> and Buster for the coming 6 months. We might build both images in CI and >>>>> add a new "matrix" dimension and run most tests on Bullseye but some >>>>> tests on Buster (specifically MSSQL until Bullseye is supported). >>>>> >>>>> We can publish both types of images (with -buster, -bullseye suffixes >>>>> same as Python) but the default image should be changed to Bullseye. >>>>> >>>>> This will add some complexity (and build overhead) to our CI, but I >>>>> think it's worth it. >>>>> >>>>> Then we could drop support and release the -buster images after it >>>>> reaches end of life (but we will leave the users possibility of building >>>>> their own buster images without guarantee that it will work). >>>>> >>>>> I think this might become our Policy also for the future (6 months >>>>> before end-of-life we switch by default to the new stable and provide >>>>> built images until end-of-life). >>>>> >>>>> WDYT? Does it look like a good proposal? >>>>> >>>>> J. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>
