On Fri, Oct 4, 2019 at 11:02 AM Robert Bradshaw <rober...@google.com> wrote:
> Thanks for holding this vote. Note that this is a pledge to remove > support sometime in 2020, but no promises as to whether that will be > January or December (though I hope sooner rather than later) Right. > Valentyn, did you want to go ahead and make a PR adding Apache Beam to > the python3statement page? > Yes, I sent https://github.com/python3statement/python3statement.github.io/pull/265. > > On Mon, Sep 30, 2019 at 5:10 PM Valentyn Tymofieiev <valen...@google.com> > wrote: > > > > As suggested and enthusiastically supported by several folks in this > thread, I will send a vote to sign a pledge on http://python3statement.org > on behalf of Apache Beam to discontinue Python 2 support in or before 2020. > > > > The motivation for signing the pledge is: > > - to provide another signal to Beam users, and projects that depend on > Beam that Beam Python 2 offering will soon sunset; > > - to facilitate adoption of Python 3 by Beam users, developers, and > runner maintainers; > > - to facilitate adoption of Python 3 in wider Python ecosystem. > > > > See also http://python3stament.org for background behind this pledge > and the list of projects which have already signed it. > > > > On Mon, Sep 23, 2019 at 4:45 PM Kyle Weaver <kcwea...@google.com> wrote: > >> > >> Re feedback collection, we already print a message: > >> "You are using Apache Beam with Python 2. New releases of Apache Beam > will soon support Python 3 only." > >> When users run Python 2 pipelines. This might be a good place to > provide additional info along with a place to send feedback (probably user@). > While I'm sure not everyone out there reads their logs, I imagine this is a > sure and easy way of reaching at least some Python 2 users. > >> > >> Kyle Weaver | Software Engineer | github.com/ibzib | > kcwea...@google.com > >> > >> > >> On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev < > valen...@google.com> wrote: > >>> > >>> Thank you, Chad, for refreshing this conversation and adding the > perspective of Python 2 users of Beam who have not(yet) completed the > migration. My thoughts below. > >>> > >>> - It is in the best interest of everyone to ensure a smooth migration > for Beam users. However a migration needs to happen since Python ecosystem > is moving off of Python 2. > >>> - Beam has a couple of dozen dependencies, and we cannot have an > expectation that Python 2 versions of these dependencies will be maintained > in 2020. > >>> - BEAM-1251 should be closed, since it may communicate a signal that > Beam does not support Python 3, while it does. Beam has first announced > support of Python 3 in Beam 2.11.0, admittedly later than many mainstream > libraries in Python ecosystem. > >>> - I think Python 2 LTS release (if we continue them) may have critical > bug fixes, but not new features, so we won't be backporting new features. > >>> - Beam portability allows users to customize usercode runtime > environment, and it should be possible for users to supply a Python 2 SDK > harness container, should they have no other option. This would require a > backported user-supplied version of Beam SDK that works on Python 2, > although such SDK may become difficult/impractical to maintain for most > users. > >>> - There are several open issues related to Python 3, but they are > improvements in nature, and we are steadily closing them off. I am not > aware of any adoption blockers for Beam Python 3, specific to Beam. > >>> - I have not heard of users reports who attempted but were not able to > use Beam on Python 3. > >>> - This does not mean that our offering is perfect, there may be errors > and omissions that are yet to be discovered. However, it would be in the > best interest of the Beam community to discover these issues earlier. A > message that Beam will discontinue Python 2 support will encourage users to > migrate, therefore I also support Beam signing > https://python3statement.org. > >>> - Having more usage statistics and feedback closer to 2020 can help us > be more confident in deciding when to stop Python 2 support. > >>> > >>> On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay <al...@google.com> wrote: > >>>> > >>>> Thanks a lot for sharing your thoughts, I completely agree that we > need to minimize the burden on our users as much as possible. Especially in > this case when we are offering a robust python 3 solution just now. However > I do share the same concerns related to dependencies and tool chains, It > will be increasingly difficult for us to keep our code base compatible with > python2 and python3 overtime. (To be very explicit, one of those > dependencies is Dataflow's python pre-portability workers.) > >>>> > >>>> On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels <m...@apache.org> > wrote: > >>>>> > >>>>> Granted that we just have finalized the Python 3 support, we should > >>>>> allow time for it to mature and for users to make the switch. > >>>>> > >>>>> > Oh, and one more thing, I think it'd make sense for Apache Beam to > >>>>> > sign https://python3statement.org/. The promise is that we'd > >>>>> > discontinue Python 2 support *in* 2020, which is not committing us > to > >>>>> > January if we're not ready. Worth a vote? > >>>>> > >>>>> +1 > >>>> > >>>> > >>>> +1 > >>>> > >>>>> > >>>>> > >>>>> On 19.09.19 15:59, Robert Bradshaw wrote: > >>>>> > Oh, and one more thing, I think it'd make sense for Apache Beam to > >>>>> > sign https://python3statement.org/. The promise is that we'd > >>>>> > discontinue Python 2 support *in* 2020, which is not committing us > to > >>>>> > January if we're not ready. Worth a vote? > >>>>> > > >>>>> > > >>>>> > On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw < > rober...@google.com> wrote: > >>>>> >> > >>>>> >> Exactly how long we support Python 2 depends on our users. Other > than > >>>>> >> those that speak up (such as yourself, thanks!), it's hard to get > a > >>>>> >> handle on how many need Python 2 and for how long. (Should we > send out > >>>>> >> a survey? Maybe after some experience with 2.16?) > >>>> > >>>> > >>>> +1, we had some success with collecting information from users using > Twitter surveys. > >>>> > >>>>> > >>>>> >> > >>>>> >> On the one hand, the whole ecosystem is finally moving on, and > even if > >>>>> >> Beam continues to support Python 2 our dependencies, or other > projects > >>>>> >> that are being used in conjunction with Beam, will also be going > >>>>> >> Python 3 only. On the other hand, Beam is, admittedly, quite late > to > >>>>> >> the party and could be the one holding people back, and looking > at how > >>>>> >> long it took us, if we just barely make it by the end of the year > it's > >>>>> >> unreasonable to say at that point "oh, and we're dropping 2.7 at > the > >>>>> >> same time." > >>>>> >> > >>>>> >> The good news is that 2.16 is shaping up to be a release I would > >>>>> >> recommend everyone migrate to Python 3 on. The remaining issues > are > >>>>> >> things like some issues with main sessions (which already has > issues > >>>>> >> in Python 2) and not supporting keyword-only arguments (a new > feature, > >>>>> >> not a regression). I would guess that even 2.15 is already good > enough > >>>>> >> for most people, at least to kick the tires and running tests to > start > >>>>> >> the effort. > >>>> > >>>> > >>>> I share the same sentiment. Beam 2.16 will offer a strong python 3 > offering. Yes, there are known issues but this is not much different than > the known issues for rest of the python offering. > >>>> > >>>>> > >>>>> >> > >>>>> >> (I also agree with the sentiment that once we go 3.x only, it'll > be > >>>>> >> likely harder to maintain a 2.x LTS... but the whole LTS thing is > >>>>> >> being discussed in another thread.) > >>>>> > >>>>> >> > >>>>> >> On Thu, Sep 19, 2019 at 2:44 PM Chad Dombrova <chad...@gmail.com> > wrote: > >>>>> >>> > >>>>> >>> Hi all, > >>>>> >>> I had a read through this thread in the archives. It occurred > before I joined the mailing list, so I hope that this email connects up > with the thread properly for everyone. > >>>>> >>> > >>>>> >>> I'd like to respond to the following points: > >>>>> >>> > >>>>> >>>> I believe we are referring to two separate things with support: > >>>>> >>>> - Supporting existing releases for patches - I agree that we > need to give > >>>>> >>>> users a long enough window to upgrade. Great if it happens with > an LTS > >>>>> >>>> release. Even if it does not, I think it will be fair to offer > patches on > >>>>> >>>> the last python 2 supporting release during some part of 2020 > if that > >>>>> >>>> becomes necessary. > >>>>> >>>> - Making new releases with python 2 support - Each new Beam > release with > >>>>> >>>> python 2 support will implicitly extend the lifetime of beam's > python 2 > >>>>> >>>> support. I do not think we need to extend this to beyond 2019. > 2 releases > >>>>> >>>> (~ 3 months) after solid python 3 support will very likely put > the last > >>>>> >>>> python 2 supporting release to last quarter of 2019 already. > >>>>> >>> > >>>>> >>> > >>>>> >>> With so many important features still under active development > (portability, expansion, external IO transforms, schema coders) and new > versions of executors tied to the Beam source, staying behind is not really > an option for many of us, and with python3 support not yet fully completed, > the window in which Beam is fully working for both python versions is > rapidly approaching 2 months, and could ultimately be even less, depending > on how long it takes to complete the dozen remaining issues in Jira, and > whatever pops up thereafter. > >>>>> >>> > >>>>> >>>> The cost of maintaining Python 2.7 support is higher than 0. > Some issues > >>>>> >>>> that come to mind: > >>>>> >>>> - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase > makes it > >>>>> >>>> difficult to use Python 3 syntax in Beam which may be necessary > to support > >>>>> >>>> and test syntactic constructs introduced in Python 3. > >>>>> >>>> - Running additional test suites increases the load on test > infrastructure > >>>>> >>>> and increases flakiness. > >>>>> >>> > >>>>> >>> > >>>>> >>> I would argue that the cost of maintaining a python2-only LTS > version will be far greater than maintaining python2 support for a little > while longer. Dropping support for python2 could mean a number of things > from simply disabling the python2 tests, to removing 2-to-3 idioms in favor > of python3-only constructs. If what you have in mind is anything like the > latter then the master branch will become quite divergent from the LTS > release, and backporting changes will be not be as simple as cherry-picking > commits. All-in-all, I think it's a lose/lose for everyone -- users and > developers, of which I am both -- to drop python2 support on such a short > timeline. > >>>>> >>> > >>>>> >>> I'm an active contributor to this project and it will put me and > the company that I work for in a very bad position if you force us onto an > LTS release in early 2020. I understand the appeal of moving to > python3-only code and I want to get there too, but I would hope that you > give your users are much time to transition their own code as the Beam > project itself has taken. I'm not asking for a full 12 months to > transition, but more than a couple will be required. > >>>> > >>>> > >>>> What would be the ideal time frame for you? > >>>> > >>>>> > >>>>> >>> > >>>>> >>> thanks, > >>>>> >>> -chad > >>>>> >>> > >>>>> >>> > >>>>> >>> > >>>>> >>> >
