> Also, what's our longer term plan for MySQL 5.7 once it reaches EOL? The same we did with Python/Kubernetes - we will drop support for it. We will stop running tests on CI and remove documentation and parts of code that is specifically there to address 5.7 and the next (minor) release after that will have no 5.7 support.
J. On Tue, Jul 26, 2022 at 6:44 PM Ping Zhang <pin...@umich.edu> wrote: > Hi Jarek, > > Thanks for bringing this up. It is a great topic to discuss. > > I like the first 2 options. For the 3rd option in the UI, I think it > might be too much as the Airflow UI is heavily used by DE/DS, but not the > admin of the clusters, or it can be in the Admin tab? > > Also, what's our longer term plan for MySQL 5.7 once it reaches EOL? > > Thanks, > > Ping > > > On Sun, Jul 24, 2022 at 8:44 AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> Hello everyone, >> >> TL;DR; I wanted to propose to start treating MySQL 5.7 in a special way >> (and add warning about upcoming disabling of support for Older DBs users >> use). We have still more than year of support for MySQL 5.7 (October 2023) >> by Oracle, but mySQL 5.7 has some behaviours that make it less stable, and >> less scalable and causes some deadlock problems. >> >> # Why do I think it is needed? >> >> We have (not a huge by repeating) consistent reports from our users who >> are reporting deadlocks on MySQL. Some of them we fixed, some of them are a >> bit mysterious, but some of them are there because we rely a bit too much >> on "SKIP_LOCKED" functionality in some cases (mini-scheduler) or because >> users - despite warnings try to run multiple schedulers on 5.7: >> >> Examples - at least SOME of the deadlocks reported here are almost for >> sure result of mini-scheduler running in parallel to main scheduler: >> https://github.com/apache/airflow/issues/16982 >> The problem is that with 5.7 we silently skip "SKIP_LOCKED" and this >> simply leads to occasional Deadlocks. >> >> # What do I propose ? >> >> My proposal would be to implement a few "guards" and "warnings" to >> prevent those cases from happening (rather than lose our time on >> false-reports). There are some legitimate deadlocks we should track down, >> but if we see a number of deadlocks which come from the 5.7 limitations, >> it's a bit hard to weed them out (not every user will share their version >> and often users simply piggyback writing "I have the same issue" and will >> not tell their version. Also I think it would be rather better for >> Airflow's reputation that they do not experience the "Deadlock" issues in >> the first place. We have an easy way to detect if SKIP_LOCKED is supported >> which we use already. >> >> The proposals: >> >> 1) Create a lock at startup of the scheduler, that will prevent from >> starting another scheduler). This can be a global lock (similar as we do in >> migration). We can test if the lock is set and fail the second scheduler if >> one is already running with appropriate message ("please upgrade to MySQL 8 >> if you want to use multiple schedulers").. Should be very easy. >> >> 2) Hard-disable mini-scheduler (no matter if it is enabled or not) for >> MySQL 5.7. Should be easy. >> >> 3) Check if we are running MySQL5.7 and warn the user (both in logs and >> in the UI) that time of live for the database they use is coming and they >> should consider an update. This we could do in general - Postgres 10 end of >> life is coming much faster - November 2022) so that could be a generally >> useful notification to everyone I think. We could also add some extra >> information in case of MySQL 5.7 that it does not support all the features >> of Airflow and that it's better to upgrade anyway. I'd treat MySQL 5.7 a >> bit less and run such a deprecation notice a year before it happens due to >> that (for Postgres, I think ~ 6 months should be fine). >> >> WDYT. >> >> J. >> >> >> >> >> >> >>
