Re: Production Grade GitOps Based Kubernetes Setup

2021-08-10 Thread Svend
Hi all, I reached out [1] to Filipe Regadas, the author of the Spotify fork of the GCP k8s operator you linked in your message. He confirms he's actively working on it and would welcome PR and community input. I have a modest PR I'll submit to him some time this week already. This seems to me

Re: Production Grade GitOps Based Kubernetes Setup

2021-08-09 Thread Niklas Wilcke
Hi Yuval, thank you for sharing all the information. I forgot to mention the Lyft operator. Thanks for "adding" it to the list. About the dual cluster approach during upgrade I have some doubts about the resource usage. If you are operating some "big" jobs that would mean you always have to pro

Re: Production Grade GitOps Based Kubernetes Setup

2021-08-09 Thread Niklas Wilcke
Hi Maciek, thanks for sharing your insights. It is highly appreciated. Regards, Niklas UNIBERG GmbH Simon-von-Utrecht-Straße 85a 20359 Hamburg niklas.wil...@uniberg.com Mobile: +49 160 9793 2593 Office: +49 40 2380 6523 UNIBERG GmbH, Dorfstraße 3, 23816 Bebensee Registergericht / Register

Re: Production Grade GitOps Based Kubernetes Setup

2021-08-06 Thread Yuval Itzchakov
Hi Niklas, We are currently using the Lyft operator for Flink in production ( https://github.com/lyft/flinkk8soperator), which is additional alternative. The project itself is pretty much in Zombie state, but commits happen every now and then. 1. Native Kubernetes could definitely work with GitOp

Re: Production Grade GitOps Based Kubernetes Setup

2021-08-06 Thread Maciej Bryński
Hi Niklas, We had the same problem one year ago and we choose Ververica Platform Community Edttion. Pros: - support for jobs on Session Clusters - good support for restoring jobs from checkpoints and savepoints - support for even hundreds of jobs Cons: - state in SQLite (we've already corrupted db

Production Grade GitOps Based Kubernetes Setup

2021-08-06 Thread Niklas Wilcke
Hi Flink Community, I'm currently assessing the situation about how to properly deploy Flink on Kubernetes via GitOps. There are some options available to deploy Flink on Kubernetes, which I would like to discuss. In general we are looking for an open source or at least unpaid solution, but I