My point is that a PR is enough for this to my mind. > On 24 Oct 2025, at 21:43, Jarek Potiuk <[email protected]> wrote: > > Still I think we change things we explicitly told our users they can use > ... if someone asks, it should not be a single person decision, it should > be something we as community agreed to - if only no-one opposed. > > It's not "Ash said it's ok" or "Jarek said it's ok". As a community we > agreed to it. > > This is what decision making in the ASF project looks like. Governance is > important. > > J > > On Fri, Oct 24, 2025 at 10:35 PM Ash Berlin-Taylor <[email protected]> wrote: > >> I don’t think even a lazy consensus is needed. >> >> If someone wants to use the Mysql client library specifically, let them >> build their own docker image from scratch. We don’t have to, nor can we, >> cater for every possible whim in our docker images. >> >> The Astronomer Runtime images have been built using >> https://packages.debian.org/trixie/default-libmysqlclient-dev which >> gives us mariadb (and the same on bookworm etc) for years, and none of our >> users have noticed (and yes, it _is_ being used, I checked that) >> >> Cheers, >> -ash >> >>> On 24 Oct 2025, at 18:41, Jarek Potiuk <[email protected]> wrote: >>> >>> If there are no comments or objections - I will start a lazy consensus >>> about it on Monday. MySQL client is already removed in main (to unblock >>> CI) - so if lazy consensus passes, it will stay like that. >>> >>> On Thu, Oct 23, 2025 at 12:56 PM Jarek Potiuk <[email protected]> wrote: >>> >>>>> And if there are issues, would we be able to fix them in Airflow? >>>> >>>> No idea - we had no issues of that sort. Yet it's the third time we >>>> have issues with Oracle drivers that we actually know we are not able to >>>> fix without Oracle fixing their side. >>>> >>>> On Thu, Oct 23, 2025 at 12:52 PM Jarek Potiuk <[email protected]> wrote: >>>> >>>>> PR to remove mysql client is here - >>>>> https://github.com/apache/airflow/pull/57146 . We should merge it now >> to >>>>> fix our canary CI builds. We can revert it later if we will find ways >> to >>>>> make MySQL work properly (though I fail to see how it can be achieved >> while >>>>> Oracle does what it does). >>>>> >>>>> On Thu, Oct 23, 2025 at 12:45 PM Jarek Potiuk <[email protected]> >> wrote: >>>>> >>>>>>> What’s the current compatibility situation between MySQL and MariaDB? >>>>>> Would the OSS client be able to connect to Oracle-provided MySQL >> server >>>>>> software? And if there are issues, would we be able to fix them in >> Airflow? >>>>>> >>>>>> As explained above - for two years CI and PROD images of Airflow use >>>>>> MariaDB client. There were zero issues reported about compatibility. >>>>>> >>>>>> On Thu, Oct 23, 2025 at 12:37 PM Tzu-ping Chung via dev < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> What’s the current compatibility situation between MySQL and MariaDB? >>>>>>> Would the OSS client be able to connect to Oracle-provided MySQL >> server >>>>>>> software? And if there are issues, would we be able to fix them in >> Airflow? >>>>>>> >>>>>>> >>>>>>> >>>>>>>> On 23 Oct 2025, at 18:25, Jarek Potiuk <[email protected]> wrote: >>>>>>>> >>>>>>>> Hello Everyone, >>>>>>>> >>>>>>>> *TL;DR; I would like to propose complete dropping of MySQL "Oracle >>>>>>>> published" client libraries from our container images.* >>>>>>>> >>>>>>>> Two years ago [1] - we switched to MariaDB clients by default >> because >>>>>>> of a >>>>>>>> very convoluted (and plain wrong) approach of Oracle for their Apt >>>>>>>> repositories and we are back to the situation we faced 2 years ago. >>>>>>>> >>>>>>>> We protected (nicely) against total disaster (where I had to >> manually >>>>>>> build >>>>>>>> and push 100 of our broken images during the weekend) and switched >> to >>>>>>>> MariaDB by default. We still left the option to build the image with >>>>>>> MySQL >>>>>>>> client and we still run it in our CI - this is how we found tonight >>>>>>>> that the problem is back, Luckily all that is needed is we need to >>>>>>> drop the >>>>>>>> optional support we have for MySQL images described in [2]. It >>>>>>> requires the >>>>>>>> users who wish to use MySQL client to build the images using our >>>>>>>> Dockerfiles with specific build arguments. We kept it for >>>>>>> compatibility and >>>>>>>> convenience of those who would have to use the clients. We never >>>>>>> heard back >>>>>>>> from anyone if they are using or not - it's very likely, it's used >>>>>>>> extremely rarely (if at all). >>>>>>>> >>>>>>>> The problem is (and you can find many articles, stack overflow >>>>>>> issues, blog >>>>>>>> posts about it) that Oracle uses a very convoluted and wrong way of >>>>>>> making >>>>>>>> their apt packages available - they sign their packages and repos >> with >>>>>>>> expiring keys. No other company I know is using this, this is >> against >>>>>>>> debian recommendations and every two years it causes the same >> problem >>>>>>> - the >>>>>>>> old packages are not installable, images released in the past that >>>>>>> have >>>>>>>> their repos added are blocked from installing **anything** (i.e. apt >>>>>>>> install fails to install anything unless you remove oracle's repos >> and >>>>>>>> keys) >>>>>>>> >>>>>>>> Just to be clear - this is (so far) not a problem for the server >> side >>>>>>> of >>>>>>>> MySQL. We are ok with our tests where MySQL is used as a server - >>>>>>> because >>>>>>>> we can use images they publish to run the servers) and MariaDB >>>>>>> clients work >>>>>>>> well with those. But for MySQL clients - every 2 years (it's already >>>>>>> the >>>>>>>> 3rd time it happened) it makes our images and dockerfiles broken - >>>>>>> and our >>>>>>>> users who want to use the clients - scrambling to install those. >>>>>>>> >>>>>>>> To add to that - when it happens, Oracle is surprised. Always. No >>>>>>>> exception. It happened tonight and as of tonight, you have no way >>>>>>>> installing the packages at all (even if you somehow get hold of the >>>>>>> new >>>>>>>> key) because their repos are signed with old, expired keys - 2 years >>>>>>> ago it >>>>>>>> took them almost a week to fix it and the bug created [3] was >> created >>>>>>> where >>>>>>>> I - among others explained them the problem they had and what >>>>>>> solution they >>>>>>>> can apply. >>>>>>>> >>>>>>>> They ignored it. Today the story repeated itself. MySQL clients >>>>>>> stopped >>>>>>>> installing - because they are signed by expired keys and their repo >>>>>>> is also >>>>>>>> signed by the same expired key. They have learned nothing and did >> not >>>>>>> fix >>>>>>>> the problem. They will have another sh**tstom coming and will >>>>>>> scramble to >>>>>>>> fix it again. >>>>>>>> >>>>>>>> But I do not wish our community (and me particularly) to be part of >>>>>>> it any >>>>>>>> more - my proposal is to simply drop that option and let the users >>>>>>> be on >>>>>>>> their own if they want to use MySQL client. >>>>>>>> >>>>>>>> I will proceed with removing it now in a PR (completely) so that we >>>>>>> fix our >>>>>>>> failing canary builds and If no-one objects, I will call for LAZY >>>>>>> CONSENSUS >>>>>>>> and will not revert the change. >>>>>>>> >>>>>>>> J. >>>>>>>> >>>>>>>> [1] Lazy consensus from 2023 >>>>>>>> https://lists.apache.org/thread/rxbyxg11jg7y35k8om0f8wgb2l9h459l >>>>>>>> [2] Optional support for MySQL clients >>>>>>>> >>>>>>> >> https://airflow.apache.org/docs/docker-stack/build.html#building-images-with-mysql-client >>>>>>>> [3] Bug from 2 years ago - https://bugs.mysql.com/bug.php?id=113432 >>>>>>> >>>>>>> >>>>>>> --------------------------------------------------------------------- >>>>>>> To unsubscribe, e-mail: [email protected] >>>>>>> For additional commands, e-mail: [email protected] >>>>>>> >>>>>>> >> >>
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
