I can tell that for us at Wix - migrating to Postgres is not going to be simple nor fast. We get alot of services from our DBA team because we are using MySQL. Think of it like a partially/semi managed Airflow. Removing support for a database feels to me like something we should give a heads up. This is not just a feature we are removing. Companies may have work to do in "preparing the ground" before they can actually migrate so if we are notifying about dropping MySQL support only when we release Airflow 3 this may be a problem.
On Sat, Nov 6, 2021 at 12:41 PM Jarek Potiuk <ja...@potiuk.com> wrote: > Just to be clear. IF we get rid of MySQL this means no support for > MariaDB either. > > On Sat, Nov 6, 2021 at 11:30 AM Alexandre Vermeerbergen > <avermeerber...@gmail.com> wrote: > > > > To be more precise: > > +1 to get MySQL out > > +1 to get MariaDB official support - we're running Airflow with > > MariaDB without troubles and we're reluctant to move to Postgres, as > > we have no admin skills on that later and lots on MariaDB > > > > Le sam. 6 nov. 2021 à 11:24, Alexandre Vermeerbergen > > <avermeerber...@gmail.com> a écrit : > > > > > > +1 let the Elephant get out from the room - a jungle in a better place > > > for Elephants that rooms :) > > > > > > Le sam. 6 nov. 2021 à 11:18, Jarek Potiuk <ja...@potiuk.com> a écrit : > > > > > > > > Hey everyone, > > > > > > > > Some of us had a discussion about MariaDB support here > > > > https://github.com/apache/airflow/pull/18506 and as a result I think > > > > this might be a good time to talk about the Elephant in the room we > > > > have. > > > > > > > > I would like to know what others think about the potential of > REMOVING > > > > MySQL support in future Airflow versions ? > > > > > > > > I believe for quite some time MySQL is the "Elephant in the room" for > > > > us, and it's one of the things that already slows us down when we add > > > > new features and when at some point we start thinking about Airflow > 3, > > > > maybe, just maybe we could think about removing support for it. > > > > > > > > Why thinking about removing MySQL? > > > > > > > > Quoting the quote of Kaxil from our discussion " "Do less but do them > > > > well". We are relying more and more on more sophisticated features > and > > > > queries of the underlying DB and this has already hit - especially > the > > > > people who developed new features but also those who helped others > > > > with issues. > > > > > > > > There are multiple problems with MySQL: deadlocks, encoding problems, > > > > support for different query constructs we have and they keep on > > > > reappearing. I personally developed quite negative feelings for MySQL > > > > while working on Airflow. > > > > > > > > Some more context: > > > > > > > > * All the Airflow-As-A-Service providers are using Postgres now as > of Airflow 2. > > > > * It seems from some discussions with people - that migration from > > > > MySQL to Postgres is possible and we could even develop a tool for > > > > that for users who would like to migrate in Airflow 3. > > > > * We also have MsSQL - which is fresh but I think there might be > > > > stronger reasons for people to use it - especially if they are in > > > > Azure/MS "world" (but we could also consider dropping it as well) > > > > * I do not think there are "super-strong" reasons why people would > > > > like to stick to MySQL. Yes, there are people who prefer it - but in > > > > our case the DB is really an "internal" piece of Airflow. I can > > > > imagine people use Postgres only for Airflow even if for the majority > > > > of other things they use MySQL. > > > > * MySQL was 25% last time we checked: > > > > https://airflow.apache.org/blog/airflow-survey-2020/ but I bet a > lot > > > > of that was Composer 1.* (Which with Airflow 2 is gone). > > > > > > > > I wonder what others think? > > > > > > > > J. >
