On Tue, Feb 7, 2023 at 2:35 PM Anand Inguva <ananding...@google.com> wrote:
> Yes, it is related to protobuf only. But I think the update of these > dependencies are required for Python 3.11 since the newer versions have > support for Python 3.11 wheels. > Assuming you refer to protobuf. Yes, there are no wheels for 3.10 for protobuf==3.x.x and that can cause friction. https://pypi.org/project/protobuf/3.20.3/#files I would probably narrow the problem further to demonstrate which stubs are not being generated, and if reason not obvious we can also ask for feedback from protobuf maintainers. Also - do we by chance need to update some other deps from https://github.com/apache/beam/blob/master/sdks/python/build-requirements.txt#L28-L33 for this to work? Also: tracking issue for protobuf4 support in Beam: https://github.com/apache/beam/issues/24569. If we use older versions of these packages, then we have to depend on > installing those packages on Python 3.11 from source distributions which is > not desired. > > I am working parallely on that issue in a different PR > https://github.com/apache/beam/pull/24599 but I think this issue should > be a blocker for Python 3.11 update. > > On Tue, Feb 7, 2023 at 5:25 PM Valentyn Tymofieiev <valen...@google.com> > wrote: > >> Hi Anand, >> >> On Tue, Feb 7, 2023 at 1:35 PM Anand Inguva via dev <dev@beam.apache.org> >> wrote: >> >>> Hi all, >>> >>> We are planning to work on adding support for Python 3.11[1] to Apache >>> Beam Python SDK. >>> >>> As part of this effort, we are going to update the python build >>> dependencies defined at [2]. >>> >>> Right now, there is an error with the newer version of >>> protobuf(4.21.11). It is not generating _urn files. >>> >>> It can be reproduced by >>> >> >>> 1. python setup.py sdist >>> 2. pip install dist/apache-beam-x.xx.x.dev0.tar.gz >>> 3. switch to python interpreter and run import apache_beam as beam >>> >> I think the error you are describing is related to protobuf 4, so the >> repro should focus on the portion where generation of stubs is happening. >> Presumably some stubs are not generated on protobuf 4 + Python 3.11? >> >> >>> >>> will lead to *ImportError: cannot import name >>> 'beam_runner_api_pb2_urns' from 'apache_beam.portability.api'. *Running >>> `python gen_protos.py` to forcefully generate files didn't help either. >>> >>> If you have encountered this error and found a resolution, please let me >>> know(that would be super helpful). >>> >>> I am going to work on this soon. Please let me know if you want to >>> collaborate. >>> >>> Thanks, >>> Anand Inguva >>> >>> *[1] *https://github.com/apache/beam/pull/24721 >>> [2] >>> https://github.com/apache/beam/blob/master/sdks/python/build-requirements.txt >>> >>
