Re: Reply:DelegationTokenManager

2022-06-21 Thread Gabor Somogyi
Thanks for pinging me! Yes, this is my main target to finish this feature however there are major code parts which are still missing. Please have a look at the umbrella jira to get better understanding: https://issues.apache.org/jira/browse/FLINK-21232 In general it's not advised to use it for pro

Re: Flink Kubernetes Operator with K8S + Istio + mTLS - port definitions

2022-06-21 Thread Elisha, Moshe (Nokia - IL/Kfar Sava)
Hi, Thank you all for your replies. The suggestion “to allow Akka cluster communication to bypass the Istio sidecar proxy” helped and we were able to deploy. I understand and agree with the rational to “follow a standard as Flink and avoid implementing guidelines from different vendors/provide

Flink k8s Operator on AWS?

2022-06-21 Thread Matt Casters
Hi Flink team! I'm interested in getting the new Flink Kubernetes Operator to work on AWS EKS. Following the documentation I got pretty far. However, when trying to run a job I got the following error: Only "local" is supported as schema for application mode. This assumes t > hat the jar is loc

Re: Flink k8s Operator on AWS?

2022-06-21 Thread Őrhidi Mátyás
Hi Matt, - In FlinkDeployments you can utilize an init container to download your artifact onto a shared volume, then you can refer to it as local:/.. from the main container. FlinkDeployments comes with pod template support https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/do

Re: Flink k8s Operator on AWS?

2022-06-21 Thread Gyula Fóra
A small addition to what Matyas has said: The limitation of only supporting local scheme is coming from the Flink Kubernetes Application mode directly and is not related to the operator itself. Once this feature is added to Flink itself the operator can also support it for newer Flink versions. G

Re: Flink k8s Operator on AWS?

2022-06-21 Thread Matt Casters
Thank you very much for the help Matyas and Gyula! I just saw a video today where you were presenting the FKO. Really nice stuff! So I'm guessing we're executing "flink run" at some point on the master and that this is when we need the jar file to be local? Am I right in assuming that this happe

Re: Flink k8s Operator on AWS?

2022-06-21 Thread Őrhidi Mátyás
Hi Matt, I believe an artifact fetcher (e.g https://hub.docker.com/r/agiledigital/s3-artifact-fetcher ) + the pod template ( https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/docs/custom-resource/pod-template/#pod-template) is an elegant way to solve your problem. The operato

Re: Flink k8s Operator on AWS?

2022-06-21 Thread Yang Wang
Matyas and Gyula have shared many great informations about how to make the Flink Kubernetes Operator work on the EKS. One more input about how to prepare the user jars. If you are more familiar with K8s, you could use persistent volume to provide the user jars and them mount the volume to JobManag

[ANNOUNCE] Apache Flink 1.14.5 released

2022-06-21 Thread Xingbo Huang
The Apache Flink community is very happy to announce the release of Apache Flink 1.14.5, which is the fourth bugfix release for the Apache Flink 1.14 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: [ANNOUNCE] Apache Flink 1.14.5 released

2022-06-21 Thread Martijn Visser
Thank you Xingbo and our community for creating this release! Op wo 22 jun. 2022 om 05:51 schreef Xingbo Huang : > The Apache Flink community is very happy to announce the release of Apache > Flink 1.14.5, which is the fourth bugfix release for the Apache Flink 1.14 > series. > > Apache Flink® is