Does flink support multi cluster deployment with HA on multi k8s clusters

2021-04-28 Thread bhagi@R
Hi Team, production requirement is to deploy flink in multi cluster mode, i.e deploying flink cluster1 with HA on kubernetes cluster1 in data center1 & another flink cluster2 with HA on kubernetes cluster2 in data center2 .. if Flink cluster1 goes down on k8s cluster1 on DC1 ,it has to fail over

Re: Does flink support multi cluster deployment with HA on multi k8s clusters

2021-04-29 Thread Till Rohrmann
Hi Bhagi, out of the box, Flink does not support this functionality. What you could try to do is to configure a stretch K8s cluster running on different DCs and then having an HA service (ZK or K8s) and blob storage being able to survive a DC outage. That way Flink should also be able to survive a

Re: Does flink support multi cluster deployment with HA on multi k8s clusters

2021-04-29 Thread bhagi@R
Can you explain clearly... How to configure HA service with k8s for different DC's for Flink cluster .. -- Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/

Re: Does flink support multi cluster deployment with HA on multi k8s clusters

2021-04-29 Thread Enrique
Hi Till and Bhagi, As part of our product, we also have a production requirement to deploy Flink in multi zones and currently have it configured with Kubernetes HA. Till, is the reason this is not supported out of the box also due to the fact Kubernetes HA relies on RWX storage and this is consid

Re: Does flink support multi cluster deployment with HA on multi k8s clusters

2021-04-29 Thread Till Rohrmann
The reason why this feature is not part of Flink is that it is not trivial to solve. There are definitely different solutions to the problem. I think that a solution will probably be built around Flink and not integrated into it because it goes a bit beyond what Flink's current focus is. I can tell