PR to remove mysql client is here - https://github.com/apache/airflow/pull/57146 . We should merge it now to fix our canary CI builds. We can revert it later if we will find ways to make MySQL work properly (though I fail to see how it can be achieved while Oracle does what it does).
On Thu, Oct 23, 2025 at 12:45 PM Jarek Potiuk <[email protected]> wrote: > > What’s the current compatibility situation between MySQL and MariaDB? > Would the OSS client be able to connect to Oracle-provided MySQL server > software? And if there are issues, would we be able to fix them in Airflow? > > As explained above - for two years CI and PROD images of Airflow use > MariaDB client. There were zero issues reported about compatibility. > > On Thu, Oct 23, 2025 at 12:37 PM Tzu-ping Chung via dev < > [email protected]> wrote: > >> What’s the current compatibility situation between MySQL and MariaDB? >> Would the OSS client be able to connect to Oracle-provided MySQL server >> software? And if there are issues, would we be able to fix them in Airflow? >> >> >> >> > On 23 Oct 2025, at 18:25, Jarek Potiuk <[email protected]> wrote: >> > >> > Hello Everyone, >> > >> > *TL;DR; I would like to propose complete dropping of MySQL "Oracle >> > published" client libraries from our container images.* >> > >> > Two years ago [1] - we switched to MariaDB clients by default because >> of a >> > very convoluted (and plain wrong) approach of Oracle for their Apt >> > repositories and we are back to the situation we faced 2 years ago. >> > >> > We protected (nicely) against total disaster (where I had to manually >> build >> > and push 100 of our broken images during the weekend) and switched to >> > MariaDB by default. We still left the option to build the image with >> MySQL >> > client and we still run it in our CI - this is how we found tonight >> > that the problem is back, Luckily all that is needed is we need to drop >> the >> > optional support we have for MySQL images described in [2]. It requires >> the >> > users who wish to use MySQL client to build the images using our >> > Dockerfiles with specific build arguments. We kept it for compatibility >> and >> > convenience of those who would have to use the clients. We never heard >> back >> > from anyone if they are using or not - it's very likely, it's used >> > extremely rarely (if at all). >> > >> > The problem is (and you can find many articles, stack overflow issues, >> blog >> > posts about it) that Oracle uses a very convoluted and wrong way of >> making >> > their apt packages available - they sign their packages and repos with >> > expiring keys. No other company I know is using this, this is against >> > debian recommendations and every two years it causes the same problem - >> the >> > old packages are not installable, images released in the past that have >> > their repos added are blocked from installing **anything** (i.e. apt >> > install fails to install anything unless you remove oracle's repos and >> > keys) >> > >> > Just to be clear - this is (so far) not a problem for the server side of >> > MySQL. We are ok with our tests where MySQL is used as a server - >> because >> > we can use images they publish to run the servers) and MariaDB clients >> work >> > well with those. But for MySQL clients - every 2 years (it's already the >> > 3rd time it happened) it makes our images and dockerfiles broken - and >> our >> > users who want to use the clients - scrambling to install those. >> > >> > To add to that - when it happens, Oracle is surprised. Always. No >> > exception. It happened tonight and as of tonight, you have no way >> > installing the packages at all (even if you somehow get hold of the new >> > key) because their repos are signed with old, expired keys - 2 years >> ago it >> > took them almost a week to fix it and the bug created [3] was created >> where >> > I - among others explained them the problem they had and what solution >> they >> > can apply. >> > >> > They ignored it. Today the story repeated itself. MySQL clients stopped >> > installing - because they are signed by expired keys and their repo is >> also >> > signed by the same expired key. They have learned nothing and did not >> fix >> > the problem. They will have another sh**tstom coming and will scramble >> to >> > fix it again. >> > >> > But I do not wish our community (and me particularly) to be part of it >> any >> > more - my proposal is to simply drop that option and let the users be >> on >> > their own if they want to use MySQL client. >> > >> > I will proceed with removing it now in a PR (completely) so that we fix >> our >> > failing canary builds and If no-one objects, I will call for LAZY >> CONSENSUS >> > and will not revert the change. >> > >> > J. >> > >> > [1] Lazy consensus from 2023 >> > https://lists.apache.org/thread/rxbyxg11jg7y35k8om0f8wgb2l9h459l >> > [2] Optional support for MySQL clients >> > >> https://airflow.apache.org/docs/docker-stack/build.html#building-images-with-mysql-client >> > [3] Bug from 2 years ago - https://bugs.mysql.com/bug.php?id=113432 >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >>
