How did you start spark job server and what version of Apache Beam SDK did you use?
There were some protocol changes recently so if both versions are not matched you may see gRPC errors. If you used the gradle command on the latest head for starting spark job server, I would recommend checking out the same version of the source with the SDK version you installed and trying again. On Wed, May 13, 2020 at 2:51 PM Naveen M <navav1...@gmail.com> wrote: > Hi, > > I am trying to run sample WordCount beam job with PortableRunner by > following the documentation here, > > https://beam.apache.org/documentation/runners/spark/ > > I want to run this as a spark-submit command with YARN resource manager. > > Can you please let me know what is missing here? Thanks your help. > > > I tried the below commands and giving some weird errors, > > > > > spark-submit --master yarn --deploy-mode client --driver-memory 2g > --executor-memory 1g --executor-cores 1 WordCount.py --input "XXXX" > --output "XXXX" --runner PortableRunner --job_endpoint localhost:8099 > > > > > > > File "/usr/local/lib64/python3.7/site-packages/apache_beam/pipeline.py", > line 503, in __exit__ > > self.run().wait_until_finish() > > File "/usr/local/lib64/python3.7/site-packages/apache_beam/pipeline.py", > line 483, in run > > self._options).run(False) > > File "/usr/local/lib64/python3.7/site-packages/apache_beam/pipeline.py", > line 496, in run > > return self.runner.run_pipeline(self, self._options) > > File > "/usr/local/lib64/python3.7/site-packages/apache_beam/runners/portability/portable_runner.py", > line 384, in run_pipeline > > job_service_plan.submit(proto_pipeline) > > File > "/usr/local/lib64/python3.7/site-packages/apache_beam/runners/portability/portable_runner.py", > line 105, in submit > > prepare_response.staging_session_token) > > File > "/usr/local/lib64/python3.7/site-packages/apache_beam/runners/portability/portable_runner.py", > line 190, in stage > > staging_location='') > > File > "/usr/local/lib64/python3.7/site-packages/apache_beam/runners/portability/stager.py", > line 229, in stage_job_resources > > self.stage_artifact(pickled_session_file, staged_path) > > File > "/usr/local/lib64/python3.7/site-packages/apache_beam/runners/portability/portable_stager.py", > line 98, in stage_artifact > > self._artifact_staging_stub.PutArtifact(artifact_request_generator()) > > File "/usr/local/lib64/python3.7/site-packages/grpc/_channel.py", line > 1011, in __call__ > > return _end_unary_response_blocking(state, call, False, None) > > File "/usr/local/lib64/python3.7/site-packages/grpc/_channel.py", line > 729, in _end_unary_response_blocking > > raise _InactiveRpcError(state) > > grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated > with: > > status = StatusCode.UNIMPLEMENTED > > details = "Method not found: > org.apache.beam.model.job_management.v1.ArtifactStagingService/PutArtifact" > > debug_error_string = > "{"created":"@1589406258.175447016","description":"Error received from peer > ipv6:[::1]:8098","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Method > not found: > org.apache.beam.model.job_management.v1.ArtifactStagingService/PutArtifact","grpc_status":12}" >
