[
https://issues.apache.org/jira/browse/BEAM-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997438#comment-16997438
]
Chad Dombrova commented on BEAM-8966:
-------------------------------------
hmmm... the errors show two different problems.
In Udi's, mypy-protobuf is newly installed, protoc-gen-mypy is found by
protoc, but there's a failure to import google.protobuf. In Kamil's
mypy-protobuf is already installed, but protoc-gen-mypy is not found by protoc.
Does anyone have a way to reproduce this reliably? What about locally?
I have a hunch this is the kind of problem that would be solved by the pep517
efforts I've been working on. In the meantime, the mypy tests are not required
– they don't actually pass yet, they're simply there for reference – so we
could remove the mypy-protobuf change and disable the tests until after the
pep517 changes.
> failure in :sdks:python:test-suites:direct:py37:hdfsIntegrationTest
> -------------------------------------------------------------------
>
> Key: BEAM-8966
> URL: https://issues.apache.org/jira/browse/BEAM-8966
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Udi Meiri
> Assignee: Chad Dombrova
> Priority: Major
>
> I believe this is due to https://github.com/apache/beam/pull/9915
> {code}
> [0mCollecting mypy-protobuf==1.12
> Using cached
> https://files.pythonhosted.org/packages/b6/28/041dea47c93564bfc0ece050362894292ec4f173caa92fa82994a6d061d1/mypy_protobuf-1.12-py3-none-any.whl
> Installing collected packages: mypy-protobuf
> Successfully installed mypy-protobuf-1.12
> [91mbeam_fn_api.proto: warning: Import google/protobuf/descriptor.proto but
> not used.
> beam_fn_api.proto: warning: Import google/protobuf/wrappers.proto but not
> used.
> [0m[91mTraceback (most recent call last):
> File "/usr/local/bin/protoc-gen-mypy", line 13, in <module>
> import google.protobuf.descriptor_pb2 as d
> ModuleNotFoundError: No module named 'google'
> [0m[91m--mypy_out: protoc-gen-mypy: Plugin failed with status code 1.
> [0m[91mProcess Process-1:
> Traceback (most recent call last):
> File "/app/sdks/python/gen_protos.py", line 104, in generate_proto_files
> from grpc_tools import protoc
> ModuleNotFoundError: No module named 'grpc_tools'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in
> _bootstrap
> self.run()
> File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run
> self._target(*self._args, **self._kwargs)
> File "/app/sdks/python/gen_protos.py", line 189, in
> _install_grpcio_tools_and_generate_proto_files
> generate_proto_files()
> File "/app/sdks/python/gen_protos.py", line 144, in generate_proto_files
> '%s' % ret_code)
> RuntimeError: Protoc returned non-zero status (see logs for details): 1
> [0m[91mTraceback (most recent call last):
> File "/app/sdks/python/gen_protos.py", line 104, in generate_proto_files
> from grpc_tools import protoc
> ModuleNotFoundError: No module named 'grpc_tools'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "setup.py", line 295, in <module>
> 'mypy': generate_protos_first(mypy),
> File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line
> 145, in setup
> return distutils.core.setup(**attrs)
> File "/usr/local/lib/python3.7/distutils/core.py", line 148, in setup
> dist.run_commands()
> File "/usr/local/lib/python3.7/distutils/dist.py", line 966, in run_commands
> self.run_command(cmd)
> File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
> cmd_obj.run()
> File "/usr/local/lib/python3.7/site-packages/setuptools/command/sdist.py",
> line 44, in run
> self.run_command('egg_info')
> File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
> self.distribution.run_command(command)
> File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
> cmd_obj.run()
> File "setup.py", line 220, in run
> gen_protos.generate_proto_files(log=log)
> File "/app/sdks/python/gen_protos.py", line 121, in generate_proto_files
> raise ValueError("Proto generation failed (see log for details).")
> ValueError: Proto generation failed (see log for details).
> [0mService 'test' failed to build: The command '/bin/sh -c cd sdks/python &&
> python setup.py sdist && pip install --no-cache-dir $(ls
> dist/apache-beam-*.tar.gz | tail -n1)[gcp]' returned a non-zero code: 1
> {code}
> https://builds.apache.org/job/beam_PostCommit_Python37/1114/consoleText
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
