Re: Deploying with Flink and Beam Python Worker

2021-01-14 Thread Sam Bourne
Thanks! I actually managed to have my own deployment for running it locally and it works well for local file, but I get this weird error while trying to run the word count example and I’m trying to understand what can be the cause of it? If you’re referring to this error: 2021/01/14 14:11:45

Re: Deploying with Flink and Beam Python Worker

2021-01-14 Thread Nir Gazit
BTW - is this something we may want to add to the official repo? (I can do it ofc). It took me a while to find how to set up a local deployment for Flink and the docs weren't super detailed. On Thu, Jan 14, 2021 at 9:39 PM Nir Gazit wrote: > Thanks! I actually managed to have my own deployment

Re: Deploying with Flink and Beam Python Worker

2021-01-14 Thread Nir Gazit
Thanks! I actually managed to have my own deployment for running it locally and it works well for local file, but I get this weird error while trying to run the word count example and I'm trying to understand what can be the cause of it? On Thu, Jan 14, 2021 at 7:29 PM Sam Bourne wrote: > Hi

Re: Deploying with Flink and Beam Python Worker

2021-01-14 Thread Sam Bourne
Hi Nir, I have a simple repo where I have a proof of concept deployment setup for doing this. https://github.com/sambvfx/beam-flink-k8s Depending on the type of runner you're using there are a few explanations. That repo should hopefully point you in the right direction. On Thu, Jan 14, 2021

Deploying with Flink and Beam Python Worker

2021-01-14 Thread Nir Gazit
Hi, I'm trying to deploy the word count job on a Flink cluster in Kubernetes. However, when trying to run the job (with python workers as a side car to the Flink task masters), I get the following error: 2021/01/14 14:11:36 Initializing python harness: /opt/apache/beam/boot --id=1-1