Not sure if this is done already, but I think places with "ODBC Driver 17 for SQL Server" hardcoded in might need to change from 17 to 18 as well: https://github.com/apache/airflow/search?q=ODBC+Driver+17+for+SQL+Server
On Wed, Feb 16, 2022 at 12:06 AM Jarek Potiuk <[email protected]> wrote: > Yep. Already did :) > > On Wed, Feb 16, 2022 at 6:22 AM Alex Begg <[email protected]> wrote: > >> I wanted to point out that not only did Microsoft add Debian Bullseye >> support for MSSQL today, to do so they did a major version >> upgrade, from msodbcsql17 to msodbcsql18. So in addition to updating the >> URL used to populate mssql-release.list, you also need to change to >> msodbcsql18. I see msodbcsql17 is in multiple places in the Airflow code >> base. >> >> Instructions for installing msodbcsql18 in Debian are here: >> >> https://docs.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15#debian18 >> >> Alex >> >> >> On Tue, Feb 15, 2022 at 1:32 AM Jarek Potiuk <[email protected]> wrote: >> >>> Yeah. That would definitely be a simpler approach for us. >>> >>> What we can also do is we can still make sure that you can actually >>> "build" Buster images if you want and we could run the simple tests we >>> already have for those images (those tests just check very basic stuff >>> - like if expected packages are importable, if Airflow commands work, >>> if you can run airflow as root user etc.. >>> This would be very little to no complication of the CI and we could >>> provide a recipe for the users how to build their own buster image - >>> but without guarantees that everything works there. >>> >>> This way might provide the users with a "workable solution" while we >>> could keep it simple for us and put a strong emphasis on "If you want >>> to get full support - also upgrade to the latest stable OS". >>> >>> J. >>> >>> >>> On Tue, Feb 15, 2022 at 9:53 AM Ash Berlin-Taylor <[email protected]> >>> wrote: >>> > >>> > My vote is to just move forward and only release Bullseye images for >>> 2.3 onwards >>> > >>> > On 15 February 2022 08:43:50 GMT, Jarek Potiuk <[email protected]> >>> wrote: >>> >> >>> >> Good news from today - I got the response from Microsoft and it seems >>> >> that MSSQL Bullseye support is going to be released today :) . So at >>> >> least that problem is going to be solved. Still the question of >>> >> whether we should build/release both Bullseye and Buster ? >>> >> >>> >> J. >>> >> >>> >> On Mon, Feb 14, 2022 at 2:14 PM Jarek Potiuk <[email protected]> >>> wrote: >>> >>> >>> >>> >>> >>> And Python default images are Bullseye now. >>> >>> >>> >>> On Mon, Feb 14, 2022 at 2:13 PM Jarek Potiuk <[email protected]> >>> wrote: >>> >>>> >>> >>>> >>> >>>> Buster end of life is August 2022 of course :) >>> >>>> >>> >>>> On Mon, Feb 14, 2022 at 2:12 PM Jarek Potiuk <[email protected]> >>> wrote: >>> >>>>> >>> >>>>> >>> >>>>> Hello everyone, >>> >>>>> >>> >>>>> I am just about to complete preparation to make our images (and >>> CI) work on Debian 11 images (Bullseye) vs Debian 10 (Buster). All looks >>> (almost) good, but I would like to tap into the collective wisdom of the >>> group here to decide on next steps (and maybe propose a policy we might be >>> following in the future). The last change that allows to easily switch >>> between the two is in reviee: >>> https://github.com/apache/airflow/pull/21546 >>> >>>>> >>> >>>>> Context/where we are: >>> >>>>> >>> >>>>> For quite a while we used Buster (Debian 10) as our base image. >>> Buster is a stable release and it is being replaced with the next stable >>> release Bullseye (Debian 11). >>> >>>>> >>> >>>>> The Release schedule is here: >>> https://wiki.debian.org/DebianReleases >>> >>>>> >>> >>>>> The most important facts from those: >>> >>>>> * Bullseye was introduced in August 2021 (6 months ago) and has >>> no end-of-life yet >>> >>>>> * Buster was introduced in July 2019 and its end of life is ~ >>> August 2020 (approx. 6 months from now) >>> >>>>> >>> >>>>> Python default images are (as of a few months), However they >>> provide (as usual -buster and -bullseye) specific images. >>> >>>>> >>> >>>>> The Bullseye switch is really needed to support ARM (M1) images >>> because of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=989604 >>> (still unresolved on Buster). >>> >>>>> >>> >>>>> It seems appropriate to switch to Bullseye now, however there is >>> one problem with it - MSSQL drivers do not yet support Bullseye. I've >>> reached out to the maintainers and await their answer when it might be >>> available: >>> https://github.com/MicrosoftDocs/sql-docs/issues/7255#issuecomment-1037097131 >>> . This is the last blocker that prevents full switch (failing PR for a full >>> switch is https://github.com/apache/airflow/pull/21378). All tests pass >>> except MSSQL. >>> >>>>> >>> >>>>> But I think we need to proceed, regardless of the timeline and >>> switch to Bullseye. >>> >>>>> >>> >>>>> Question: >>> >>>>> >>> >>>>> We need to decide on what support we give our users in the images >>> now, in the 2.3.0 release and in the future. We have no policies for that >>> yet, so it might be a good time to decide now. Some of the users might >>> "rely" on the Buster being used by the images as they might have some >>> incompatible libraries (similarly to MSSQL). And I believe we need to >>> support Buster till at least the end of life of it. >>> >>>>> >>> >>>>> This support might be twofold: >>> >>>>> >>> >>>>> * we can publish in https://hub.docker.com/r/apache/airflow our >>> binary images with Buster and/or Bullseye >>> >>>>> * we can only publish one of those (and change from Buster to >>> Bullseye for 2.3.0) but users will be able to build their own custom images >>> using our Dockerfile for either of those. >>> >>>>> >>> >>>>> Proposal: >>> >>>>> >>> >>>>> I think there are multiple reasons why we should support both >>> Bullseye and Buster for the coming 6 months. We might build both images in >>> CI and add a new "matrix" dimension and run most tests on Bullseye but some >>> tests on Buster (specifically MSSQL until Bullseye is supported). >>> >>>>> >>> >>>>> We can publish both types of images (with -buster, -bullseye >>> suffixes same as Python) but the default image should be changed to >>> Bullseye. >>> >>>>> >>> >>>>> This will add some complexity (and build overhead) to our CI, but >>> I think it's worth it. >>> >>>>> >>> >>>>> Then we could drop support and release the -buster images after >>> it reaches end of life (but we will leave the users possibility of building >>> their own buster images without guarantee that it will work). >>> >>>>> >>> >>>>> I think this might become our Policy also for the future (6 >>> months before end-of-life we switch by default to the new stable and >>> provide built images until end-of-life). >>> >>>>> >>> >>>>> WDYT? Does it look like a good proposal? >>> >>>>> >>> >>>>> J. >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>> >>
