Hello everyone, With the upcoming merge of the Google libraries update ( https://github.com/apache/airflow/pull/30067) where Google Team and developers they work with made a Herculean effort to upgrade some 20 client libraries, and few other recent dependency changes, we should be very close to finally (after more 6 months!) move to Python 3.11
I kept the PR open since the release time and tracked what's left https://github.com/apache/airflow/pull/27264 The two remaining pieces are Nteract's Papermill and Cloudera's Python-SSL (the latter iis needed for apache.hive provider). Both have PRs that someone created to bring 3.11 support - but they are not merged/released yet. So as the first step in provider suspension process I let the maintainers of both libraries that they are holding us back and informed that we might decide to suspend the Papermill/apache.hive providers from releases, according to our process: * Papermill https://github.com/nteract/papermill/issues/700#issuecomment-1549633305 * Cloudera's Python-ssl: https://github.com/cloudera/python-sasl/issues/30#issuecomment-1549643700 Before formally voting/asing for consensus to suspend those providers I will explore other options (maybe it will be enough to disable them for 3.11), but I just wanted to let everyone know that it **might** happen. I am personally looking forward to being able to release Airflow 2.7 with Python 3.11 (and without Python 3.7) support. Python 3.11 on its own can give ~30% improvement of speed mostly coming from specializing adaptive interpreters and tons of other speed improvements. So being able to officially support Airflow + most providers for Python 3.11 is cool. (many of us already used it for development, but until most of the important Providers are ready it makes little sense to release Airflow claiming "3.11" support. I hope move to 3.12 will be way faster, there are some things coming in 3.12 that improve multi-cpu performance for in-process multiple interpreters (per-interpreter GIL) which I am looking forward to, it MIGHT help us with improving multi-CPU capabilities of Airflow without having to use forking and multiple processes, which might actually help quite a bit with DAG parsing performance and some of our caching woes (but this is a just a guess that we **might** be able to use it). And the final shape of 3.12 will only be known in October. J.
