I do not yet want to cast a formal vote on it, because maybe others have other ideas, but let me formulate what I see as a good proposal now.
The "default" in our case means mainly about the one that is used by "non-approved yet PR" (we will continue running tests for all supported versions in master and similarly as yesterday we wil be able to fix them quickly). So I think we can be a bit more aggressive on the "default" version. Just to show an example of what can happen - we had master failing yesterday for Python 3.8 (fixed here - https://github.com/apache/airflow/pull/12481) and I guess we might have more cases like that. The proposal from Damian was to keep up with the schedule of Python releases. I would like to - slightly - modify it to use the latest supported version as "default" to prevent the kind of failures. " I think about such set of rules: * we use the "latest" supported version of Python as the default (3.8 today). * we officially support all Python releases till the end of their EOL We can always decide to support versions past EOL if we have a good reason - but it would have to be justified and voted. How does it sound? Any other proposals ? Just to remind the Python schedule: Branch Schedule Status First release End-of-life 3.9 PEP 596 bugfix 2020-10-05 TBD 3.8 PEP 569 bugfix 2019-10-14 2024-10 3.7 PEP 537 security 2018-06-27 2023-06-27 3.6 PEP 494 security 2016-12-23 2021-12-23 J. -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/> On Tue, Nov 17, 2020 at 8:19 PM Leah Cole <[email protected]> wrote: > I really like Damian's proposal. With my google devrel work we had been > doing something similar to what Jarek had done above - looking at PyPI > downloads and at what is most popular, but doing that makes it harder to > encourage folks to move towards the future. I like that Python provides > this consistent schedule - it will make it easier for us to set > expectations with our users, and will align with other Python-y things. :) > And, tbh, probably going to propose this in future meetings related to my > devrel work so thank you Damien! > > On Thu, Nov 12, 2020 at 6:46 AM Jarek Potiuk <[email protected]> > wrote: > >> I love the idea to have clear rules and tie it to the official schedule >> of Python releases - at least as a target, because we might find some >> issues that might prevent us from doing so. >> >> Also we should officially support all versions that are not yet reached >> end-of-life. >> >> I like the proposed schedule and yearly cadence. I wonder if others have >> similar thoughts. Such agreement/policy would require formal voting though >> I think? >> >> WDYT everyone? >> >> J. >> >> >> On Thu, Nov 12, 2020 at 3:37 PM Shaw, Damian P. < >> [email protected]> wrote: >> >>> I just wanted to add that if people are not aware PEP 0602 >>> <https://www.python.org/dev/peps/pep-0602> has been accepted and >>> implemented for Python 3.9. This means 3 things for the Python release >>> cycle: >>> >>> 1. A new version every 12 months >>> >>> 2. Each version receives 18 months of full support (bug fixes and >>> security fixes) >>> >>> 3. After full support has ended each version receives an >>> additional 42 months of security updates >>> >>> >>> >>> Going forward I think it makes sense to bump up the default version of >>> Python every 1 year in cadence with the Python release cycle. Assuming >>> people agreed the question would be how far behind should Airflow be from >>> the new release? >>> >>> >>> >>> Personally I feel like no more than 18 months is a good, in the new >>> Python release cadence that version of Python will no longer be receiving >>> bug fixes and therefore will be very stable, and 18 months is a good enough >>> time for any libraries and providers to be available (if they’re not >>> available after 18 months maybe they have given up support?) >>> >>> >>> >>> If we retroactively apply this to the previous releases of Python that >>> would put us at Python 3.7 default now and Python 3.8 default ~April 14, >>> 2021. >>> >>> >>> >>> My 2 cents, >>> >>> Damian >>> >>> >>> >>> >>> >>> *From:* Jarek Potiuk <[email protected]> >>> *Sent:* Thursday, November 12, 2020 09:03 >>> *To:* [email protected] >>> *Subject:* Re: Default/supported Python versions for Airlfow 2.0 >>> >>> >>> >>> Should we make Python 3.7 default then and leave all others as-is ? >>> >>> >>> >>> J. >>> >>> >>> >>> >>> >>> On Thu, Nov 5, 2020 at 1:48 PM Kaxil Naik <[email protected]> wrote: >>> >>> We should definitely support Python 3.6 to make the Upgrades to Airflow >>> 2.0 a bit easier. >>> >>> >>> >>> As of yesterday, checks these stats from PyPI downloads: >>> >>> >>> >>> Py3.7: 12,578 >>> >>> Py3.6: 9,806 >>> >>> Py3.8: 1,815 >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Nov 5, 2020 at 11:40 AM Halo Ku <[email protected]> wrote: >>> >>> >>> >>> If I may point that Airflow is a wokrflow managment system and as >>> such the power of the tool is in direct extention to the levrage providers. >>> This should also be checked from how many of the providers are >>> compatible with 3.8 / 3.9 >>> >>> >>> >>> *Sent:* Thursday, November 05, 2020 at 1:16 PM >>> *From:* "Ash Berlin-Taylor" <[email protected]> >>> *To:* "[email protected]" <[email protected]> >>> *Cc:* "[email protected]" <[email protected]> >>> *Subject:* Re: Default/supported Python versions for Airlfow 2.0 >>> >>> Debian stable ships python 3.7(.3) >>> >>> CentOS 8 has two packages - python36 and python38 >>> >>> Ubuntu 18.04 (LTS) has 3.6.5 >>> >>> Ubuntu 20.04 (LTS) has 3.8.2 >>> >>> >>> >>> (https://pkgs.org/search/?q=python3&on=files) >>> >>> >>> >>> RHEL is harder to find out about . RHEL8 has python 3.6 as python3, and >>> RHEL 8.2 has Py3.8 as a separate package >>> https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_basic_system_settings/index#using-python3_configuring-basic-system-settings >>> >>> >>> >>> So for default version 3.8 or 3.9 gets my vote. I think the cost/burden >>> of supporting back to 3.6 is not very great, so we should continue to >>> support (and I guess test) that. >>> >>> >>> >>> -ash >>> >>> >>> >>> On Nov 5 2020, at 8:49 am, Jarek Potiuk <[email protected]> >>> wrote: >>> >>> Hello Everyone, >>> >>> >>> >>> I have a question. What do people think about default version of >>> Pyhon for Airflow 2.0 (and set of supported versions)? >>> >>> >>> >>> Currently, we have python 3.6 as default, but all the version up to 3.8 >>> are officially supported and tested and PR for python 3.9 is in Draft: >>> https://github.com/apache/airflow/pull/11950 >>> >>> >>> >>> This is the release schedule for python versions. We have a year till >>> the end of 3.6 >>> >>> >>> >>> Branch Schedule Status First release End-of-life >>> >>> 3.9 PEP 596 bugfix 2020-10-05 TBD >>> >>> 3.8 PEP 569 bugfix 2019-10-14 2024-10 >>> >>> 3.7 PEP 537 security 2018-06-27 2023-06-27 >>> >>> 3.6 PEP 494 security 2016-12-23 2021-12-23 >>> >>> >>> >>> WDYT? >>> >>> >>> >>> J. >>> >>> >>> >>> >>> >>> -- >>> >>> *Jarek Potiuk* >>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>> >>> M: +48 660 796 129 <+48%20660%20796%20129> >>> [image: Polidea] <https://www.polidea.com/> >>> >>> >>> >>> >>> -- >>> >>> *Jarek Potiuk* >>> Polidea <https://www.polidea.com/> | Principal Software Engineer >>> >>> >>> >>> M: +48 660 796 129 <+48660796129> >>> [image: Polidea] <https://www.polidea.com/> >>> >>> >>> >>> >>> ============================================================================== >>> Please access the attached hyperlink for an important electronic >>> communications disclaimer: >>> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html >>> >>> ============================================================================== >>> >> >> >> -- >> >> Jarek Potiuk >> Polidea <https://www.polidea.com/> | Principal Software Engineer >> >> M: +48 660 796 129 <+48660796129> >> [image: Polidea] <https://www.polidea.com/> >> >> > > -- > > Leah Cole (she/her) | Developer Programs Engineer | [email protected] | > (925) > 257-2112 > *I'm working weird hours during this pandemic and am sometimes a bit > slower to respond to PRs/CLs than normal. Please feel free to send me a > gentle ping for a status update if my slowness is blocking you and I'll do > my best to give you an ETA. * > > -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
