That's good news! Thanks for sharing. Another datapoint, here are a few of Beam's dependencies that no longer release new py2 artifacts (I looked at REQUIRED_PACKAGES + aws, gcp, and interactive extras):
hdfs numpy pyarrow ipython There are more if we include transitive dependencies and test-only packages. I also remember encountering one issue last month that was broken only on Py2, which we had to go back and fix. If others have noticed frictions related to ongoing Py2 support or have updates on previously mentioned Py3 migration blockers, feel free to post them. On Fri, May 8, 2020 at 9:19 AM Robert Bradshaw <rober...@google.com> wrote: > It hasn't been 3 months yet, but I wanted to call out a milestone that > Python 3 downloads crossed the 50% threshold on pypi, if just briefly. > > On Thu, Feb 13, 2020 at 12:40 AM Ismaël Mejía <ieme...@gmail.com> wrote: > > > > > I would suggest re-evaluating this within the next 3 months again. We > need to balance between user pain/contributor pain/our ability to > continuously test with python 2 in a shifting environment. > > > > Good idea for the in 3 months evaluation, at that point also > distributions will probably be phasing out python2 by default which > definitely help in this direction. > > Thanks for updating the roadmap Ahmet > > > > > > On Thu, Feb 13, 2020 at 2:49 AM Ahmet Altay <al...@google.com> wrote: > >> > >> > >> > >> On Wed, Feb 12, 2020 at 1:29 AM Ismaël Mejía <ieme...@gmail.com> wrote: > >>> > >>> I am with Chad on this, we should probably extend it a bit more, even > if it > >>> makes us struggle a bit at least we have some workarounds as Robert > suggests, > >>> and as Chad said there are still many people playing the python 3 > catchup game, > >>> so worth to support those users. > >>> > >>> > >>> But maybe it is worth to evaluate the current state later in the year. > >> > >> > >> I would suggest re-evaluating this within the next 3 months again. We > need to balance between user pain/contributor pain/our ability to > continuously test with python 2 in a shifting environment. > >> > >>> > >>> In the > >>> meantime can someone please update our Roadmap in the website with > this info and > >>> where we are with Python 3 support (it looks not up to date). > >>> https://beam.apache.org/roadmap/ > >> > >> > >> I made a minor change to update that page ( > https://github.com/apache/beam/pull/10848). A more comprehensive update > to that page and linked ( > https://beam.apache.org/roadmap/python-sdk/#python-3-support) would still > be welcome. > >> > >>> > >>> > >>> - Ismaël > >>> > >>> > >>> On Tue, Feb 4, 2020 at 10:49 PM Robert Bradshaw <rober...@google.com> > wrote: > >>>> > >>>> On Tue, Feb 4, 2020 at 12:12 PM Chad Dombrova <chad...@gmail.com> > wrote: > >>>> >> > >>>> >> Not to mention that all the nice work for the type hints will > have to be redone in the for 3.x. > >>>> > > >>>> > Note that there's a tool for automatically converting type comments > to annotations: https://github.com/ilevkivskyi/com2ann > >>>> > > >>>> > So don't let that part bother you. > >>>> > >>>> +1, I wouldn't worry about what can be easily automated. > >>>> > >>>> > I'm curious what other features you'd like to be using in the Beam > source that you cannot now. > >>>> > >>>> I hit things occasionally, e.g. I just ran into wanting keyword-only > >>>> arguments the other day. > >>>> > >>>> >> It seems the faster we drop support the better. > >>>> > > >>>> > > >>>> > I've already gone over my position on this, but a refresher for > those who care: some of the key vendors that support my industry will not > offer python3-compatible versions of their software until the 4th quarter > of 2020. If Beam switches to python3-only before that point we may be > forced to stop contributing features (note: I'm the guy who added the type > hints :). Every month you can give us would be greatly appreciated. > >>>> > >>>> As another data point, we're still 80/20 on Py2/Py3 for downloads at > >>>> PyPi [1] (which I've heard should be taken with a grain of salt, but > >>>> likely isn't totally off). IMHO that ratio needs to be way higher for > >>>> Python 3 to consider dropping Python 2. It's pretty noisy, but say it > >>>> doubles every 3 months that would put us at least mid-year before we > >>>> hit a cross-over point. On the other hand Q4 2020 is probably a > >>>> stretch. > >>>> > >>>> We could consider whether it needs to be an all-or-nothing thing as > >>>> well. E.g. perhaps some features could be Python 3 only sooner than > >>>> the whole codebase. (This would have to be well justified.) Another > >>>> mitigation is that it is possible to mix Python 2 and Python 3 in the > >>>> same pipeline with portability, so if there's a library that you need > >>>> for one DoFn it doesn't mean you have to hold back your whole > >>>> pipeline. > >>>> > >>>> - Robert > >>>> > >>>> [1] https://pypistats.org/packages/apache-beam , and that 20% may > just > >>>> be a spike. >
