Re: Using pyflink from flink distribution

Great, thank you so much for your responses. It all makes sense now. :) On Mon, Jan 30, 2023 at 10:41 PM Dian Fu wrote: > >> What is the reason for including > opt/python/{pyflink.zip,cloudpickle.zip,py4j.zip} in the base > distribution then? Oh, a guess: to make it easier for TaskManagers to

Re: Using pyflink from flink distribution

>> What is the reason for including opt/python/{pyflink.zip,cloudpickle.zip,py4j.zip} in the base distribution then? Oh, a guess: to make it easier for TaskManagers to run pyflink without having pyflink installed themselves? Somehow I'd guess this wouldn't work tho; I'd assume TaskManagers would

Re: Using pyflink from flink distribution

Thanks Dian! > >> Is using pyflink from the flink distribution tarball (without pip) not a supported way to use pyflink? > You are right. What is the reason for including opt/python/{pyflink.zip,cloudpickle.zip,py4j.zip} in the base distribution then? Oh, a guess: to make it easier for

Re: Using pyflink from flink distribution

Hi Andrew, >> By pip installing apache-flink, this docker image will have the flink distro installed at /opt/flink and FLINK_HOME set to /opt/flink . BUT ALSO flink lib jars will be installed at e.g.

Re: Using pyflink from flink distribution

Ah, oops and my original email had a typo: > Some python dependencies are not included in the flink distribution tarballs: cloudpickle, py4j and pyflink are in opt/python. Should read: > Some python dependencies ARE included in the flink distribution tarballs: cloudpickle, py4j and pyflink are in

Re: Using pyflink from flink distribution

Let me ask a related question: We are building our own base Flink docker image. We will be deploying both JVM and python apps via flink-kubernetes-operator. Is there any reason not to install Flink in this image via `pip install apache-flink` and use it for JVM apps? -Andrew Otto Wikimedia

Using pyflink from flink distribution

Hello, I'm having quite a bit of trouble running pyflink from the default flink distribution tarballs. I'd expect the python examples to work as long as python is installed, and we've got the distribution. Some python dependencies are not included in the flink distribution tarballs: