Hi Navneeth, I think there might be some misunderstanding here. Let me try to clarify.
1) The so-called native Kubernetes support [1], which was added as an experimental feature in Flink 1.10, is not used by Ververica Platform CE nor by the Lyft K8s Operator as far as I am aware of. 2) The native Kubernetes support only supports session clusters [1] so far. Support for Job clusters work in progress for Flink 1.11. In case of a session cluster, Flink will request additional resources from Kubernetes whenever necessary to deploy a Flink Job upon submission. So, there is automatic scaling of the *cluster, *but no automatic scaling of the *jobs*. 3) There are multiple efforts within the Flink ecosystem working on auto scaling jobs. I hope this helps and please reach out if you have further questions. Best, Konstantin [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/native_kubernetes.html [2] https://ci.apache.org/projects/flink/flink-docs-release-1.10/concepts/glossary.html#flink-session-cluster On Thu, Apr 23, 2020 at 7:11 AM Navneeth Krishnan <reachnavnee...@gmail.com> wrote: > Thanks a lot Timo. I will take a look at it. But does flink automatically > scale up and down at this point with native integration? > > Thanks > > On Tue, Apr 14, 2020 at 11:27 PM Timo Walther <twal...@apache.org> wrote: > >> Hi Navneeth, >> >> it might be also worth to look into Ververica Plaform for this. The >> community edition was published recently is free of charge. It provides >> first class K8s support [1]. >> >> There is also a tutorial how to deploy it on EKS [2] (not the most >> recent one through). >> >> Regards, >> Timo >> >> [1] >> >> https://www.ververica.com/blog/announcing-ververica-platform-community-edition?utm_campaign=Ververica%20Platform%20-%20Community%20Edition&utm_content=123140986&utm_medium=social&utm_source=twitter&hss_channel=tw-2581958070 >> [2] >> >> https://www.ververica.com/blog/how-to-get-started-with-data-artisans-platform-on-aws-eks >> >> >> >> On 15.04.20 03:57, Navneeth Krishnan wrote: >> > Hi All, >> > >> > I'm very new to EKS and trying to deploy a flink job in cluster mode. >> > Are there any good documentations on what are the steps to deploy on >> EKS? >> > >> > From my understanding, with flink 1.10 running it on EKS will >> > automatically scale up and down with kubernetes integration based on >> the >> > load. Is this correct? Do I have to do enable some configs to support >> > this feature? >> > >> > How to use the lyft k8s operator when deploying on EKS? >> > >> > Thanks a lot, appreciate all the help. >> > >> > >> > >> > >> >> -- Konstantin Knauf | Head of Product +49 160 91394525 Follow us @VervericaData Ververica <https://www.ververica.com/> -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Tony) Cheng
