Python 3.11 support has been merged at https://github.com/apache/beam/pull/26121 targeting Beam 2.47.0 release.
Please let me know if you have any questions. Thanks, Anand On Tue, Feb 21, 2023 at 6:04 PM Valentyn Tymofieiev <valen...@google.com> wrote: > Thanks a lot Anand. I'll take a look at the PRs. > > On Tue, Feb 21, 2023 at 1:56 PM Anand Inguva <ananding...@google.com> > wrote: > >> I was able to spin up a PR: https://github.com/apache/beam/pull/24599 >> that updates the build dependencies of Apache Beam. >> >> Several GCP dependencies needed to be updated as well. I covered them in >> the PR: https://github.com/apache/beam/pull/24599 >> >> On Thu, Feb 9, 2023 at 3:29 PM Anand Inguva <ananding...@google.com> >> wrote: >> >>> Yes, we may need to update all of them >>> <https://github.com/apache/beam/blob/master/sdks/python/build-requirements.txt#L28-L33>. >>> I can add more information once I dig into the issue(most likely next >>> week). I will comment on my findings on the issue: >>> https://github.com/apache/beam/issues/24569 and will periodically >>> update this thread. >>> >>> On Tue, Feb 7, 2023 at 5:47 PM Valentyn Tymofieiev <valen...@google.com> >>> wrote: >>> >>>> On Tue, Feb 7, 2023 at 2:35 PM Anand Inguva <ananding...@google.com> >>>> wrote: >>>> >>>>> Yes, it is related to protobuf only. But I think the update of these >>>>> dependencies are required for Python 3.11 since the newer versions have >>>>> support for Python 3.11 wheels. >>>>> >>>> Assuming you refer to protobuf. Yes, there are no wheels for 3.10 for >>>> protobuf==3.x.x and that can cause friction. >>>> https://pypi.org/project/protobuf/3.20.3/#files >>>> >>>> I would probably narrow the problem further to demonstrate which stubs >>>> are not being generated, and if reason not obvious we can also ask for >>>> feedback from protobuf maintainers. Also - do we by chance need to >>>> update some other deps from >>>> https://github.com/apache/beam/blob/master/sdks/python/build-requirements.txt#L28-L33 >>>> for this to work? >>>> >>>> Also: tracking issue for protobuf4 support in Beam: >>>> https://github.com/apache/beam/issues/24569. >>>> >>>> If we use older versions of these packages, then we have to depend on >>>>> installing those packages on Python 3.11 from source distributions which >>>>> is >>>>> not desired. >>>>> >>>>> I am working parallely on that issue in a different PR >>>>> https://github.com/apache/beam/pull/24599 but I think this issue >>>>> should be a blocker for Python 3.11 update. >>>>> >>>>> On Tue, Feb 7, 2023 at 5:25 PM Valentyn Tymofieiev < >>>>> valen...@google.com> wrote: >>>>> >>>>>> Hi Anand, >>>>>> >>>>>> On Tue, Feb 7, 2023 at 1:35 PM Anand Inguva via dev < >>>>>> dev@beam.apache.org> wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> We are planning to work on adding support for Python 3.11[1] to >>>>>>> Apache Beam Python SDK. >>>>>>> >>>>>>> As part of this effort, we are going to update the python build >>>>>>> dependencies defined at [2]. >>>>>>> >>>>>>> Right now, there is an error with the newer version of >>>>>>> protobuf(4.21.11). It is not generating _urn files. >>>>>>> >>>>>>> It can be reproduced by >>>>>>> >>>>>> >>>>>>> 1. python setup.py sdist >>>>>>> 2. pip install dist/apache-beam-x.xx.x.dev0.tar.gz >>>>>>> 3. switch to python interpreter and run import apache_beam as beam >>>>>>> >>>>>> I think the error you are describing is related to protobuf 4, so the >>>>>> repro should focus on the portion where generation of stubs is happening. >>>>>> Presumably some stubs are not generated on protobuf 4 + Python 3.11? >>>>>> >>>>>> >>>>>>> >>>>>>> will lead to *ImportError: cannot import name >>>>>>> 'beam_runner_api_pb2_urns' from 'apache_beam.portability.api'. *Running >>>>>>> `python gen_protos.py` to forcefully generate files didn't help either. >>>>>>> >>>>>>> If you have encountered this error and found a resolution, please >>>>>>> let me know(that would be super helpful). >>>>>>> >>>>>>> I am going to work on this soon. Please let me know if you want to >>>>>>> collaborate. >>>>>>> >>>>>>> Thanks, >>>>>>> Anand Inguva >>>>>>> >>>>>>> *[1] *https://github.com/apache/beam/pull/24721 >>>>>>> [2] >>>>>>> https://github.com/apache/beam/blob/master/sdks/python/build-requirements.txt >>>>>>> >>>>>>
