Re: FLIP-6 Docker / Kubernetes

2018-05-11 Thread Gary Yao
Hi, Unfortunately, there won't be native support for Kubernetes in Flink 1.5 yet [1]. That is, you will still have to deploy a standalone session cluster and submit a job via the REST API. Can you open an issue in JIRA regarding the Blob Cache corruption? FLINK-5908 is closed already. Best,

Re: FLIP-6 Docker / Kubernetes

2018-05-07 Thread anaray
Thank You Fabian. We are looking forward for this feature, especially a way to bundle the job jar along with the container (even in taskmanagers). In production we deploy flink (1 job=1 cluster) in Docker Swarm. One of the main issue we face is related to blob download when the taskmanager fails

Re: FLIP-6 Docker / Kubernetes

2018-05-07 Thread Fabian Hueske
Hi, Most, but not all, of the FLIP-6 features will be released with Flink 1.5.0. I'm not sure if this deployment mode will be fully supported in 1.5.0. Gary (in CC) might know details here. Anyway, the deployment would work by starting the image using regular Docker/Kubernetes tools. The image