Tell me if you see any output. Anyway, here's the link to the same issue you're facing:
https://lists.apache.org/thread.html/4e8e1455916debe096de32551f9ab05853524cf282bc312cd4620d68@%3Cuser.beam.apache.org%3E The amount of issues I've encountered as a newbie is indeed troubling. On 2019/09/17 14:43:11, Tom Barber <[email protected]> wrote: > 🤣 okay I’ll look again, I assumed it just crashed in a ball of flames! > > > On 17 September 2019 at 15:39:33, Benjamin Tan ([email protected]) > wrote: > > I got this too! Did you manage to get any output? (I did) I reported this > in another thread. > > This looks like a key error when StopWorker is called. As far as I know, it > seems like the work has been processed. > > On Tue, Sep 17, 2019 at 10:31 PM Tom Barber <[email protected]> wrote: > > > Well my errors are different but still terminal: > > > > ERROR:grpc._server:Exception calling application: u'1-1' > > Traceback (most recent call last): > > File "/usr/local/lib/python2.7/dist-packages/grpc/_server.py", line 434, > > in _call_behavior > > response_or_iterator = behavior(argument, context) > > File > > "/usr/local/lib/python2.7/dist-packages/apache_beam-2.17.0.dev0-py2.7.egg/apache_beam/runners/worker/worker_pool_main.py", > > line 126, in StopWorker > > worker_process = > > self._worker_processes.pop(stop_worker_request.worker_id) > > KeyError: u'1-1' > > ERROR:grpc._server:Exception calling application: u'2-1' > > Traceback (most recent call last): > > File "/usr/local/lib/python2.7/dist-packages/grpc/_server.py", line 434, > > in _call_behavior > > response_or_iterator = behavior(argument, context) > > File > > "/usr/local/lib/python2.7/dist-packages/apache_beam-2.17.0.dev0-py2.7.egg/apache_beam/runners/worker/worker_pool_main.py", > > line 126, in StopWorker > > worker_process = > > self._worker_processes.pop(stop_worker_request.worker_id) > > KeyError: u'2-1' > > > > > > On 17 September 2019 at 14:46:12, Benjamin Tan ( > > [email protected]) wrote: > > > > If it helps, I’m using Spark 2.4.4. The Apache Beam Python library on > > master is 2.17.0-dev. > > > > > > On 17 Sep 2019, at 9:39 PM, Tom Barber <[email protected]> wrote: > > > > Cool thanks Benjamin, I’ll give it a shot. > > > > Tom > > > > > > On 17 September 2019 at 13:56:14, Benjamin Tan ( > > [email protected]) wrote: > > > > > > I encountered the exact same thing today. High five! Here’s how I managed > > to make some progress: > > > > 1. Used the master branch > > 2. Built and installed the Python SDK > > > > cd into the sdk library and python ./setup install > > > > I got some other errors but they didn’t seem to be show stoppers. > > > > On 17 Sep 2019, at 8:26 PM, Tom Barber <[email protected]> wrote: > > > > Hello folks, > > > > Day 3 of trying to get the basics going with Python & Spark 2.2.3. > > > > I’ve downgraded the spark version to 2.2.3 in the cradle build so that I > > can run jobs against it. > > > > I’ve then written this: > > > > options = PipelineOptions(["--runner=PortableRunner", > > "--job_endpoint=localhost:8099", "--environment_type=LOOPBACK"]) > > > > with beam.Pipeline(options=options) as p: > > print("hello") > > lines = p | 'Create' >> beam.Create(['hello', 'world']) > > > > Which in turn explodes with: > > > > Caused by: > > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.StatusRuntimeException: > > UNIMPLEMENTED: Method not found! > > at > > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:235) > > at > > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:216) > > at > > org.apache.beam.vendor.grpc.v1p21p0.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:141) > > at > > org.apache.beam.model.fnexecution.v1.BeamFnExternalWorkerPoolGrpc$BeamFnExternalWorkerPoolBlockingStub.startWorker(BeamFnExternalWorkerPoolGrpc.java:226) > > > > When run against python 2 and > > > > AttributeError: module 'apache_beam.coders.coders' has no > > attribute ‘VarIntCoder' > > > > When run with python 3.5. > > > > Should I just give up trying the portable runner at the moment or am I > > doing something dumb/incompatible? I’m stuck on Spark 2.2.3 so upgrading > > isn’t currently an option. > > > > Tom > > > > Spicule Limited is registered in England & Wales. Company Number: > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > All engagements are subject to Spicule Terms and Conditions of Business. > > This email and its contents are intended solely for the individual to whom > > it is addressed and may contain information that is confidential, > > privileged or otherwise protected from disclosure, distributing or copying. > > Any views or opinions presented in this email are solely those of the > > author and do not necessarily represent those of Spicule Limited. The > > company accepts no liability for any damage caused by any virus transmitted > > by this email. If you have received this message in error, please notify us > > immediately by reply email before deleting it from your system. Service of > > legal notice cannot be effected on Spicule Limited by email. > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > All engagements are subject to Spicule Terms and Conditions of Business. > > This email and its contents are intended solely for the individual to whom > > it is addressed and may contain information that is confidential, > > privileged or otherwise protected from disclosure, distributing or copying. > > Any views or opinions presented in this email are solely those of the > > author and do not necessarily represent those of Spicule Limited. The > > company accepts no liability for any damage caused by any virus transmitted > > by this email. If you have received this message in error, please notify us > > immediately by reply email before deleting it from your system. Service of > > legal notice cannot be effected on Spicule Limited by email. > > > > > > Spicule Limited is registered in England & Wales. Company Number: > > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > > All engagements are subject to Spicule Terms and Conditions of Business. > > This email and its contents are intended solely for the individual to whom > > it is addressed and may contain information that is confidential, > > privileged or otherwise protected from disclosure, distributing or copying. > > Any views or opinions presented in this email are solely those of the > > author and do not necessarily represent those of Spicule Limited. The > > company accepts no liability for any damage caused by any virus transmitted > > by this email. If you have received this message in error, please notify us > > immediately by reply email before deleting it from your system. Service of > > legal notice cannot be effected on Spicule Limited by email. > > > > -- > > > Spicule Limited is registered in England & Wales. Company Number: > 09954122. Registered office: First Floor, Telecom House, 125-135 Preston > Road, Brighton, England, BN1 6AF. VAT No. 251478891. > > > > > All engagements > are subject to Spicule Terms and Conditions of Business. This email and its > contents are intended solely for the individual to whom it is addressed and > may contain information that is confidential, privileged or otherwise > protected from disclosure, distributing or copying. Any views or opinions > presented in this email are solely those of the author and do not > necessarily represent those of Spicule Limited. The company accepts no > liability for any damage caused by any virus transmitted by this email. If > you have received this message in error, please notify us immediately by > reply email before deleting it from your system. Service of legal notice > cannot be effected on Spicule Limited by email. >
