Thanks for holding this vote. Note that this is a pledge to remove support sometime in 2020, but no promises as to whether that will be January or December (though I hope sooner rather than later).
Valentyn, did you want to go ahead and make a PR adding Apache Beam to the python3statement page? On Mon, Sep 30, 2019 at 5:10 PM Valentyn Tymofieiev <valen...@google.com> wrote: > > As suggested and enthusiastically supported by several folks in this thread, > I will send a vote to sign a pledge on http://python3statement.org on behalf > of Apache Beam to discontinue Python 2 support in or before 2020. > > The motivation for signing the pledge is: > - to provide another signal to Beam users, and projects that depend on Beam > that Beam Python 2 offering will soon sunset; > - to facilitate adoption of Python 3 by Beam users, developers, and runner > maintainers; > - to facilitate adoption of Python 3 in wider Python ecosystem. > > See also http://python3stament.org for background behind this pledge and the > list of projects which have already signed it. > > On Mon, Sep 23, 2019 at 4:45 PM Kyle Weaver <kcwea...@google.com> wrote: >> >> Re feedback collection, we already print a message: >> "You are using Apache Beam with Python 2. New releases of Apache Beam will >> soon support Python 3 only." >> When users run Python 2 pipelines. This might be a good place to provide >> additional info along with a place to send feedback (probably user@). While >> I'm sure not everyone out there reads their logs, I imagine this is a sure >> and easy way of reaching at least some Python 2 users. >> >> Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com >> >> >> On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev <valen...@google.com> >> wrote: >>> >>> Thank you, Chad, for refreshing this conversation and adding the >>> perspective of Python 2 users of Beam who have not(yet) completed the >>> migration. My thoughts below. >>> >>> - It is in the best interest of everyone to ensure a smooth migration for >>> Beam users. However a migration needs to happen since Python ecosystem is >>> moving off of Python 2. >>> - Beam has a couple of dozen dependencies, and we cannot have an >>> expectation that Python 2 versions of these dependencies will be maintained >>> in 2020. >>> - BEAM-1251 should be closed, since it may communicate a signal that Beam >>> does not support Python 3, while it does. Beam has first announced support >>> of Python 3 in Beam 2.11.0, admittedly later than many mainstream libraries >>> in Python ecosystem. >>> - I think Python 2 LTS release (if we continue them) may have critical bug >>> fixes, but not new features, so we won't be backporting new features. >>> - Beam portability allows users to customize usercode runtime environment, >>> and it should be possible for users to supply a Python 2 SDK harness >>> container, should they have no other option. This would require a >>> backported user-supplied version of Beam SDK that works on Python 2, >>> although such SDK may become difficult/impractical to maintain for most >>> users. >>> - There are several open issues related to Python 3, but they are >>> improvements in nature, and we are steadily closing them off. I am not >>> aware of any adoption blockers for Beam Python 3, specific to Beam. >>> - I have not heard of users reports who attempted but were not able to use >>> Beam on Python 3. >>> - This does not mean that our offering is perfect, there may be errors and >>> omissions that are yet to be discovered. However, it would be in the best >>> interest of the Beam community to discover these issues earlier. A message >>> that Beam will discontinue Python 2 support will encourage users to >>> migrate, therefore I also support Beam signing https://python3statement.org. >>> - Having more usage statistics and feedback closer to 2020 can help us be >>> more confident in deciding when to stop Python 2 support. >>> >>> On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay <al...@google.com> wrote: >>>> >>>> Thanks a lot for sharing your thoughts, I completely agree that we need to >>>> minimize the burden on our users as much as possible. Especially in this >>>> case when we are offering a robust python 3 solution just now. However I >>>> do share the same concerns related to dependencies and tool chains, It >>>> will be increasingly difficult for us to keep our code base compatible >>>> with python2 and python3 overtime. (To be very explicit, one of those >>>> dependencies is Dataflow's python pre-portability workers.) >>>> >>>> On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels <m...@apache.org> wrote: >>>>> >>>>> Granted that we just have finalized the Python 3 support, we should >>>>> allow time for it to mature and for users to make the switch. >>>>> >>>>> > Oh, and one more thing, I think it'd make sense for Apache Beam to >>>>> > sign https://python3statement.org/. The promise is that we'd >>>>> > discontinue Python 2 support *in* 2020, which is not committing us to >>>>> > January if we're not ready. Worth a vote? >>>>> >>>>> +1 >>>> >>>> >>>> +1 >>>> >>>>> >>>>> >>>>> On 19.09.19 15:59, Robert Bradshaw wrote: >>>>> > Oh, and one more thing, I think it'd make sense for Apache Beam to >>>>> > sign https://python3statement.org/. The promise is that we'd >>>>> > discontinue Python 2 support *in* 2020, which is not committing us to >>>>> > January if we're not ready. Worth a vote? >>>>> > >>>>> > >>>>> > On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw <rober...@google.com> >>>>> > wrote: >>>>> >> >>>>> >> Exactly how long we support Python 2 depends on our users. Other than >>>>> >> those that speak up (such as yourself, thanks!), it's hard to get a >>>>> >> handle on how many need Python 2 and for how long. (Should we send out >>>>> >> a survey? Maybe after some experience with 2.16?) >>>> >>>> >>>> +1, we had some success with collecting information from users using >>>> Twitter surveys. >>>> >>>>> >>>>> >> >>>>> >> On the one hand, the whole ecosystem is finally moving on, and even if >>>>> >> Beam continues to support Python 2 our dependencies, or other projects >>>>> >> that are being used in conjunction with Beam, will also be going >>>>> >> Python 3 only. On the other hand, Beam is, admittedly, quite late to >>>>> >> the party and could be the one holding people back, and looking at how >>>>> >> long it took us, if we just barely make it by the end of the year it's >>>>> >> unreasonable to say at that point "oh, and we're dropping 2.7 at the >>>>> >> same time." >>>>> >> >>>>> >> The good news is that 2.16 is shaping up to be a release I would >>>>> >> recommend everyone migrate to Python 3 on. The remaining issues are >>>>> >> things like some issues with main sessions (which already has issues >>>>> >> in Python 2) and not supporting keyword-only arguments (a new feature, >>>>> >> not a regression). I would guess that even 2.15 is already good enough >>>>> >> for most people, at least to kick the tires and running tests to start >>>>> >> the effort. >>>> >>>> >>>> I share the same sentiment. Beam 2.16 will offer a strong python 3 >>>> offering. Yes, there are known issues but this is not much different than >>>> the known issues for rest of the python offering. >>>> >>>>> >>>>> >> >>>>> >> (I also agree with the sentiment that once we go 3.x only, it'll be >>>>> >> likely harder to maintain a 2.x LTS... but the whole LTS thing is >>>>> >> being discussed in another thread.) >>>>> >>>>> >> >>>>> >> On Thu, Sep 19, 2019 at 2:44 PM Chad Dombrova <chad...@gmail.com> >>>>> >> wrote: >>>>> >>> >>>>> >>> Hi all, >>>>> >>> I had a read through this thread in the archives. It occurred before >>>>> >>> I joined the mailing list, so I hope that this email connects up with >>>>> >>> the thread properly for everyone. >>>>> >>> >>>>> >>> I'd like to respond to the following points: >>>>> >>> >>>>> >>>> I believe we are referring to two separate things with support: >>>>> >>>> - Supporting existing releases for patches - I agree that we need to >>>>> >>>> give >>>>> >>>> users a long enough window to upgrade. Great if it happens with an >>>>> >>>> LTS >>>>> >>>> release. Even if it does not, I think it will be fair to offer >>>>> >>>> patches on >>>>> >>>> the last python 2 supporting release during some part of 2020 if that >>>>> >>>> becomes necessary. >>>>> >>>> - Making new releases with python 2 support - Each new Beam release >>>>> >>>> with >>>>> >>>> python 2 support will implicitly extend the lifetime of beam's >>>>> >>>> python 2 >>>>> >>>> support. I do not think we need to extend this to beyond 2019. 2 >>>>> >>>> releases >>>>> >>>> (~ 3 months) after solid python 3 support will very likely put the >>>>> >>>> last >>>>> >>>> python 2 supporting release to last quarter of 2019 already. >>>>> >>> >>>>> >>> >>>>> >>> With so many important features still under active development >>>>> >>> (portability, expansion, external IO transforms, schema coders) and >>>>> >>> new versions of executors tied to the Beam source, staying behind is >>>>> >>> not really an option for many of us, and with python3 support not yet >>>>> >>> fully completed, the window in which Beam is fully working for both >>>>> >>> python versions is rapidly approaching 2 months, and could ultimately >>>>> >>> be even less, depending on how long it takes to complete the dozen >>>>> >>> remaining issues in Jira, and whatever pops up thereafter. >>>>> >>> >>>>> >>>> The cost of maintaining Python 2.7 support is higher than 0. Some >>>>> >>>> issues >>>>> >>>> that come to mind: >>>>> >>>> - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it >>>>> >>>> difficult to use Python 3 syntax in Beam which may be necessary to >>>>> >>>> support >>>>> >>>> and test syntactic constructs introduced in Python 3. >>>>> >>>> - Running additional test suites increases the load on test >>>>> >>>> infrastructure >>>>> >>>> and increases flakiness. >>>>> >>> >>>>> >>> >>>>> >>> I would argue that the cost of maintaining a python2-only LTS version >>>>> >>> will be far greater than maintaining python2 support for a little >>>>> >>> while longer. Dropping support for python2 could mean a number of >>>>> >>> things from simply disabling the python2 tests, to removing 2-to-3 >>>>> >>> idioms in favor of python3-only constructs. If what you have in mind >>>>> >>> is anything like the latter then the master branch will become quite >>>>> >>> divergent from the LTS release, and backporting changes will be not >>>>> >>> be as simple as cherry-picking commits. All-in-all, I think it's a >>>>> >>> lose/lose for everyone -- users and developers, of which I am both -- >>>>> >>> to drop python2 support on such a short timeline. >>>>> >>> >>>>> >>> I'm an active contributor to this project and it will put me and the >>>>> >>> company that I work for in a very bad position if you force us onto >>>>> >>> an LTS release in early 2020. I understand the appeal of moving to >>>>> >>> python3-only code and I want to get there too, but I would hope that >>>>> >>> you give your users are much time to transition their own code as the >>>>> >>> Beam project itself has taken. I'm not asking for a full 12 months >>>>> >>> to transition, but more than a couple will be required. >>>> >>>> >>>> What would be the ideal time frame for you? >>>> >>>>> >>>>> >>> >>>>> >>> thanks, >>>>> >>> -chad >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >>>
