[jira] [Comment Edited] (FLINK-15641) Support to start init container
[ https://issues.apache.org/jira/browse/FLINK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086760#comment-17086760 ] ouyangwulin edited comment on FLINK-15641 at 4/19/20, 5:38 AM: --- [~fly_in_gis]，in progressing. I will implement something in the doc before share it. was (Author: ouyangwuli): [~fly_in_gis]，in progressing. > Support to start init container > --- > > Key: FLINK-15641 > URL: https://issues.apache.org/jira/browse/FLINK-15641 > Project: Flink > Issue Type: New Feature > Components: Deployment / Kubernetes >Reporter: Yang Wang >Priority: Major > > >> Why do we need init container? > The init container could be used to prepare the use jars and dependencies. > Then we could always set the user image to Flink official image both for > standalone per-job on K8s or native K8s per-job. When the JobManager and > TaskManager container launched, the user jars will already exist there. I > think many users are running standalone per-job cluster in production by > using this way. > The init container only works for K8s cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15641) Support to start init container
[
https://issues.apache.org/jira/browse/FLINK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079048#comment-17079048
]
Canbin Zheng edited comment on FLINK-15641 at 4/9/20, 8:23 AM:
---
Ah, #1 solves my original question. For #2 and #3, I think you are talking
about the Application mode, for the K8s per-job, internally we compiled the job
graph on the client-side, so we need to advertise and download it via the init
Container. Currently, I am not very sure whether we should also support
client-side per job mode on K8s yet.
Back to #1, I have another concern,
{quote}The schema could "file://", "hdfs://", "s3://", etc. If it is a remote
path, then we need to start the init container to download these jars and share
them with JM/TM container via volume.
{quote}
We'd better extend the schema to support local:// as well, local:// means path
in the image, while [file://|file:///] means path on the client-sde.
Moreover, maybe the init Container support could be a seperate issue and we can
split it into several subtasks so that we can work together for it.
was (Author: felixzheng):
Ah, #1 solves my original question. For #2 and #3, I think you are talking
about the Application mode, for the K8s per-job, internally we compiled the job
graph on the client-side, so we need to advertise and download it via the init
Container. Currently, I am not very sure whether we should also support
client-side per job mode on K8s yet.
Back to #1, I have another concern,
{quote}The schema could "file://", "hdfs://", "s3://", etc. If it is a remote
path, then we need to start the init container to download these jars and share
them with JM/TM container via volume.
{quote}
We'd better extend the schema to support local:// as well, local:// means path
in the image, while file:// means path on the client-sde.
> Support to start init container
> ---
>
> Key: FLINK-15641
> URL: https://issues.apache.org/jira/browse/FLINK-15641
> Project: Flink
> Issue Type: Sub-task
> Components: Deployment / Kubernetes
>Reporter: Yang Wang
>Priority: Major
>
> >> Why do we need init container?
> The init container could be used to prepare the use jars and dependencies.
> Then we could always set the user image to Flink official image both for
> standalone per-job on K8s or native K8s per-job. When the JobManager and
> TaskManager container launched, the user jars will already exist there. I
> think many users are running standalone per-job cluster in production by
> using this way.
> The init container only works for K8s cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15641) Support to start init container
[ https://issues.apache.org/jira/browse/FLINK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060654#comment-17060654 ] ouyangwulin edited comment on FLINK-15641 at 3/17/20, 6:00 AM: --- [~fly_in_gis] As flink on k8s ,not native. I think init container has another Scenario, that when before the tm start ,It use to probe the jm pod have started。 and flink native on k8s, the tm pod start after the jm pod started. So it don't need the probe. was (Author: ouyangwuli): [~fly_in_gis] As flink on k8s ,not native. I think init container has another Scenario, that when before the tm start ,It use to probe the jm pod have started。 > Support to start init container > --- > > Key: FLINK-15641 > URL: https://issues.apache.org/jira/browse/FLINK-15641 > Project: Flink > Issue Type: Sub-task > Components: Deployment / Kubernetes >Reporter: Yang Wang >Priority: Major > > >> Why do we need init container? > The init container could be used to prepare the use jars and dependencies. > Then we could always set the user image to Flink official image both for > standalone per-job on K8s or native K8s per-job. When the JobManager and > TaskManager container launched, the user jars will already exist there. I > think many users are running standalone per-job cluster in production by > using this way. > The init container only works for K8s cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-15641) Support to start init container
[ https://issues.apache.org/jira/browse/FLINK-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029510#comment-17029510 ] Canbin Zheng edited comment on FLINK-15641 at 2/4/20 3:05 AM: -- Hi [~fly_in_gis], it seems that the init-container work could be included in Support per-job mode in kubernetes ? was (Author: felixzheng): Hi [~fly_in_gis], It seems that the init-container work could be included in Support per-job mode in kubernetes ? > Support to start init container > --- > > Key: FLINK-15641 > URL: https://issues.apache.org/jira/browse/FLINK-15641 > Project: Flink > Issue Type: Sub-task > Components: Deployment / Kubernetes >Reporter: Yang Wang >Priority: Major > > >> Why do we need init container? > The init container could be used to prepare the use jars and dependencies. > Then we could always set the user image to Flink official image both for > standalone per-job on K8s or native K8s per-job. When the JobManager and > TaskManager container launched, the user jars will already exist there. I > think many users are running standalone per-job cluster in production by > using this way. > The init container only works for K8s cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
