Re: [LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

Also - all past images (turns that we have > 100) have been patched. I have not rebuilt all the images, I just added extra layers on top - with installing the new GPG key from Oracle AND removing the mysql repository. This should prevent us from having to redo the same in 2025. I also made it qu

Re: [VOTE] Airflow Providers prepared on December 12, 2023

One more +1 needed, please :) On Thu, Dec 14, 2023 at 1:39 PM Hussein Awala wrote: > +1 (binding) for daskexecutor, docker and odbc providers > > On Thu 14 Dec 2023 at 13:37, Jarek Potiuk wrote: > > > A small reminder. Despite -1s on Google, databricks, i would love to have > > some +1 on the r

Re: Making Listeners API non-experimental

+1 -- Regards, Aritra Basu On Fri, Dec 15, 2023, 8:47 PM Vincent Beck wrote: > +1 > > On 2023/12/15 12:51:26 Jarek Potiuk wrote: > > +1. ESPECIALLY if there will be issues found that we will fix in 2.8.x :D > > > > On Fri, Dec 15, 2023 at 1:47 PM Elad Kalif wrote: > > > > > If There were no is

Re: [LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

BTW. Big shoutout to Andrey who implemented the easy switch between the clients, a few months ago (for ARM case) in the way that we could implement the switch **really** quickly. Kudos for anticipating the need :D On Fri, Dec 15, 2023 at 4:45 PM Jarek Potiuk wrote: > Hello everyone, > > This is

[LAZY CONSENSUS] Switch 2.8.0 images to use MariaDB client

Hello everyone, This is a bit hurried, but the circumstances are unusual. Even if we use MySQL and not MariaDB as our "production ready" database we've lost a bit of faith in Oracle being a good steward of MySQL apt repos. More details can be found in: https://lists.apache.org/thread/dy3fgwhoc5m

Re: Making Listeners API non-experimental

+1 On 2023/12/15 12:51:26 Jarek Potiuk wrote: > +1. ESPECIALLY if there will be issues found that we will fix in 2.8.x :D > > On Fri, Dec 15, 2023 at 1:47 PM Elad Kalif wrote: > > > If There were no issues discovered so far I am +1 in making it non > > experimental for 2.9 > > > > On Fri, Dec 1

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

PR with complete fix for "main" and "2.8.0": https://github.com/apache/airflow/pull/36243 Once it passes all the tests and we will know the solution there works, I will also start removing the repos for old released images. J On Fri, Dec 15, 2023 at 3:09 PM Jarek Potiuk wrote: > Small update

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

Small update. Oracle/ MySQL **just** resigned their repo with a new key, with a new one. The new workaround is: *RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B7B3B788A8D3785C && gpg --export "B7B3B788A8D3785C" > "/etc/apt/trusted.gpg.d/mysql.gpg"* Ho

Re: Making Listeners API non-experimental

+1. ESPECIALLY if there will be issues found that we will fix in 2.8.x :D On Fri, Dec 15, 2023 at 1:47 PM Elad Kalif wrote: > If There were no issues discovered so far I am +1 in making it non > experimental for 2.9 > > On Fri, Dec 15, 2023 at 2:45 PM Maciej Obuchowski > wrote: > > > Make it no

Re: Making Listeners API non-experimental

If There were no issues discovered so far I am +1 in making it non experimental for 2.9 On Fri, Dec 15, 2023 at 2:45 PM Maciej Obuchowski wrote: > Make it non -experimental in 2.9 - forgot to add. > > On Fri, Dec 15, 2023, 13:43 Maciej Obuchowski > wrote: > > > Hello everyone, > > The listener

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

> Isn't there the other option and that is to include the key for verification. Thus pinning the certificate / key? This way it is more safe to ignore the expiry date? Very good question. We already HAD the pinned certificate and that was part of the problem (the expired key ID is hard-coded in a

Re: Making Listeners API non-experimental

Make it non -experimental in 2.9 - forgot to add. On Fri, Dec 15, 2023, 13:43 Maciej Obuchowski wrote: > Hello everyone, > The listener plugin API - > https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/listeners.html > has been in Airflow as experminetal since 2.

Making Listeners API non-experimental

Hello everyone, The listener plugin API - https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/listeners.html has been in Airflow as experminetal since 2.3, and there have been some changes, but the feature overall has been stable and relatively unchanged since 2.7.

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

Isn't there the other option and that is to include the key for verification. Thus pinning the certificate / key? This way it is more safe to ignore the expiry date? I'm okay with all the other options just verifying if we are not overlooking anything. The reason for this is that we do not support

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

I would rather stick to the Option 2, with small modifications: By default use MariaDB libraries which compatible with MySQL, so change everywhere INSTALL_MYSQL_CLIENT_TYPE=mariadb - For x86_64 keep option to switch to Oracle MySQL libraries - For ARM it would always forced (or until Oracle provide

Re: [PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

Hey Jarek, Thank you for opening this discussion. I have been working since this morning on a solution to this problem in our Airflow Prod image, which is based on the Airflow docker file retrieved from tag 2.6.3, without duplicating the docker file in our repository. I agree that option 1) is mo

Re: [VOTE] Release Airflow 2.8.0 from 2.8.0rc3

Hello fellow Airflowers, A bug was found in rc3 that'll necessitate an rc4. I'm canceling this vote and will create 2.8.0rc4 once the issue is resolved. You can follow the discussion at https://lists.apache.org/thread/dy3fgwhoc5m2f3tpqkwqcytrs6nhjlc4 Regards, -Ephraim On Fri, 15 Dec 2023 at 07:2

[PROPOSAL] NEED URGENT FEEDBACK! MySQL Badly signed repo breaks Airflow images

Hello everyone, (sorry for capital letters but we need to act quickly). *TL;DR; Due to Oracle breaking MySQL repo signing, Together with Andrey we have a proposal to - immediately - switch to MariaDB clients in Airflow images for main and the upcoming Airflow 2.8.0 - giving the user an option to