xintongsong commented on a change in pull request #14629: URL: https://github.com/apache/flink/pull/14629#discussion_r580737573
########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) Review comment: ```suggestion Use [`kubernetes.pod-template-file`]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) ``` ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. Review comment: ```suggestion Flink allows users to define the JobManager and TaskManager pods via template files, with the advanced features (e.g. volumes, init containers, sidecar containers, etc.) that are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. ``` ########## File path: flink-kubernetes/src/main/java/org/apache/flink/kubernetes/configuration/KubernetesConfigOptions.java ########## @@ -367,6 +369,50 @@ code("FlinkKubeClient#checkAndUpdateConfigMap")) .build()); + public static final ConfigOption<String> JOB_MANAGER_POD_TEMPLATE = + key("kubernetes.pod-template-file.jobmanager") + .stringType() + .noDefaultValue() + .withFallbackKeys(KUBERNETES_POD_TEMPLATE_FILE_KEY) + .withDescription( + "Specify a local file that contains the jobmanager pod template definition. " + + "It will be used to initialize the jobmanager pod. " + + "If not explicitly configured, config option '" + + KUBERNETES_POD_TEMPLATE_FILE_KEY + + "' will be used."); + + public static final ConfigOption<String> TASK_MANAGER_POD_TEMPLATE = + key("kubernetes.pod-template-file.taskmanager") + .stringType() + .noDefaultValue() + .withFallbackKeys(KUBERNETES_POD_TEMPLATE_FILE_KEY) + .withDescription( + "Specify a local file that contains the taskmanager pod template definition. " + + "It will be used to initialize the taskmanager pod. " + + "If not explicitly configured, config option '" + + KUBERNETES_POD_TEMPLATE_FILE_KEY + + "' will be used."); + + /** + * This option is here only for documentation generation, it is the fallback key of + * JOB_MANAGER_POD_TEMPLATE and TASK_MANAGER_POD_TEMPLATE. + */ + @SuppressWarnings("unused") + public static final ConfigOption<String> KUBERNETES_POD_TEMPLATE = + key(KUBERNETES_POD_TEMPLATE_FILE_KEY) + .stringType() + .noDefaultValue() + .withDescription( + "Specify a local file that contains the pod template definition. " + + "It will be used to initialize the jobmanager and taskmanager pod. " + + "The main container should be defined with name '" + + Constants.MAIN_CONTAINER_NAME Review comment: Let's also mention the main container name for `JOB_MANAGER_POD_TEMPLATE ` and `TASK_MANAGER_POD_TEMPLATE` descriptions. ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template and not listed in the tables will be unaffected. Review comment: ```suggestion All the fields defined in the pod template that are not listed in the tables will be unaffected. ``` ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template and not listed in the tables will be unaffected. + +Please note that some new config options may be introduced to overwrite more Kubernetes fields, +your pod template may not work in the future Flink releases. +We could guarantee that `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, +`env`, `init container`, `sidecar container`, `volumes` from pod template will always be respected. + +#### Pod Metadata Review comment: Currently, the headings `Pod Metadata`, `Pod Spec`, `Container Spec` and `Pod Template Definition` are in the same level. I would suggest to have a section `Fields Overwritten by Flink` to discuss the overwritten issue. We can move the three tables as well as some texts from the first two paragraphs related to fields overwriting to this section ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template and not listed in the tables will be unaffected. + +Please note that some new config options may be introduced to overwrite more Kubernetes fields, +your pod template may not work in the future Flink releases. +We could guarantee that `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, +`env`, `init container`, `sidecar container`, `volumes` from pod template will always be respected. Review comment: ```suggestion Please note that Flink may overwrite more fields from pod templates in future releases. While it is undetermined which fields might be overwritten in future, Flink provides the following list of fields that are planned to be long-term supported: `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, `env`, `init container`, `sidecar container`, `volumes`. ``` ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template and not listed in the tables will be unaffected. + +Please note that some new config options may be introduced to overwrite more Kubernetes fields, +your pod template may not work in the future Flink releases. +We could guarantee that `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, +`env`, `init container`, `sidecar container`, `volumes` from pod template will always be respected. + +#### Pod Metadata +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Pod MetaData Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>name</td> + <td>See description </td> + <td>The JobManager pod name will be overwritten with the deployment which is defined by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-cluster-id">kubernetes.cluster-id</a>. + The TaskManager pod names will be overwritten with the pattern <code><clusterID>-<attempt>-<index></code> which is generated by Flink ResourceManager.</td> + </tr> + <tr> + <td>namespace</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-namespace">kubernetes.namespace</a></td> + <td>Both the JobManager deployment and TaskManager pods will be created in the specified namespace.</td> + </tr> + <tr> + <td>ownerReferences</td> + <td>See description</td> + <td>The owner reference of JobManager pod will be overwritten by deployment. + The deployment owner reference could be set by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-owner-reference">kubernetes.jobmanager.owner.reference</a>. + The owner reference of every TaskManager pod will be set to JobManager deployment.</td> + </tr> + <tr> + <td>annotations</td> + <td>Add the annotations from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-annotations">kubernetes.jobmanager.annotations</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-annotations">kubernetes.taskmanager.annotations</a></td> + <td>Flink will add additional annotations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>labels</td> + <td>Add the labels from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-labels">kubernetes.jobmanager.labels</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-labels">kubernetes.taskmanager.labels</a></td> + <td>Flink will add additional labels specified by the Flink configuration options.</td> + </tr> + </tbody> +</table> + +#### Pod Spec +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Pod Spec Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>imagePullSecrets</td> + <td>Add image pull secrets from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-secrets">kubernetes.container.image.pull-secrets</a></td> + <td>Flink will add additional pull secrets specified by the Flink configuration options.</td> + </tr> + <tr> + <td>nodeSelector</td> + <td>Add the node selectors from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-node-selector">kubernetes.jobmanager.node-selector</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-node-selector">kubernetes.taskmanager.node-selector</a></td> + <td>Flink will add additional node selectors specified by the Flink configuration options.</td> + </tr> + <tr> + <td>tolerations</td> + <td>Add the tolerations from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-tolerations">kubernetes.jobmanager.tolerations</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-tolerations">kubernetes.taskmanager.tolerations</a></td> + <td>Flink will add additional tolerations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>restartPolicy</td> + <td>"always" for JobManager deployment + <br> + "never" for TaskManager pod</td> + <td>The JobManager pod will always be restarted by deployment. And the TaskManager pod should not be restarted.</td> + </tr> + <tr> + <td>serviceAccount</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-service-account">kubernetes.service-account</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-service-account">kubernetes.jobmanager.service-account</a> for JobManager, + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-service-account">kubernetes.taskmanager.service-account</a> for TaskManager</td> + <td>The JobManager deployment and TaskManager pods will be created with the specified service account.</td> + </tr> + <tr> + <td>volumes</td> + <td>See description</td> + <td>Flink will add some additional internal ConfigMap volumes(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td> + </tr> + </tbody> +</table> + +#### Container Spec +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Container Spec Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>env</td> + <td>Add environment variables from <a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.master.env.{ENV_NAME}</a> + or <a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.taskmanager.env.{ENV_NAME}</a></td> + <td>Flink will add additional environment variables specified by the Flink configuration options.</td> + </tr> + <tr> + <td>image</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image">kubernetes.container.image</a></td> + <td>The image will be overwritten by flink configuration option.</td> + </tr> + <tr> + <td>imagePullPolicy</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-policy">kubernetes.container.image.pull-policy</a></td> + <td>The image pull policy will be overwritten by flink configuration option.</td> + </tr> + <tr> + <td>name</td> + <td>"flink-job-manager" or "flink-task-manager"</td> + <td>The container name will be overwritten by flink ("flink-job-manager" for the JobManager container, and "flink-task-manager" for every TaskManager container).</td> + </tr> + <tr> + <td>resources</td> + <td>Memory: value of <a href="{{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size">jobmanager.memory.process.size</a> + or <a href="{{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size">taskmanager.memory.process.size</a> + <br> + CPU: value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-cpu">kubernetes.jobmanager.cpu</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-cpu">kubernetes.taskmanager.cpu</a></td> + <td>The memory and cpu resources(including requests and limits) will be overwritten by flink. All other resources(e.g. ephemeral-storage) will be retained.</td> + </tr> + <tr> + <td>volumeMounts</td> + <td>See description</td> + <td>Flink will add some additional internal volume mounts(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td> + </tr> + </tbody> +</table> + +#### Pod Template Definition +`pod-template.yaml` +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: jobmanager-pod-template +spec: + initContainers: + - name: artifacts-fetcher + image: artifacts-fetcher:latest + # Use wget or other tools to get user jars from remote storage + command: [ 'wget', 'https://path/of/StateMachineExample.jar', '-O', '/flink-artifact/myjob.jar' ] + volumeMounts: + - mountPath: /flink-artifact + name: flink-artifact + containers: + # Do not change the main container name + - name: flink-main-container Review comment: The name of the main container is a key information. It should be mentioned in the beginning paragraph, where how this feature should be used is described in general. ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template and not listed in the tables will be unaffected. + +Please note that some new config options may be introduced to overwrite more Kubernetes fields, +your pod template may not work in the future Flink releases. +We could guarantee that `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, +`env`, `init container`, `sidecar container`, `volumes` from pod template will always be respected. + +#### Pod Metadata +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Pod MetaData Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>name</td> + <td>See description </td> + <td>The JobManager pod name will be overwritten with the deployment which is defined by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-cluster-id">kubernetes.cluster-id</a>. + The TaskManager pod names will be overwritten with the pattern <code><clusterID>-<attempt>-<index></code> which is generated by Flink ResourceManager.</td> + </tr> + <tr> + <td>namespace</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-namespace">kubernetes.namespace</a></td> + <td>Both the JobManager deployment and TaskManager pods will be created in the specified namespace.</td> + </tr> + <tr> + <td>ownerReferences</td> + <td>See description</td> + <td>The owner reference of JobManager pod will be overwritten by deployment. + The deployment owner reference could be set by <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-owner-reference">kubernetes.jobmanager.owner.reference</a>. + The owner reference of every TaskManager pod will be set to JobManager deployment.</td> + </tr> + <tr> + <td>annotations</td> + <td>Add the annotations from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-annotations">kubernetes.jobmanager.annotations</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-annotations">kubernetes.taskmanager.annotations</a></td> + <td>Flink will add additional annotations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>labels</td> + <td>Add the labels from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-labels">kubernetes.jobmanager.labels</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-labels">kubernetes.taskmanager.labels</a></td> + <td>Flink will add additional labels specified by the Flink configuration options.</td> + </tr> + </tbody> +</table> + +#### Pod Spec +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Pod Spec Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>imagePullSecrets</td> + <td>Add image pull secrets from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-secrets">kubernetes.container.image.pull-secrets</a></td> + <td>Flink will add additional pull secrets specified by the Flink configuration options.</td> + </tr> + <tr> + <td>nodeSelector</td> + <td>Add the node selectors from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-node-selector">kubernetes.jobmanager.node-selector</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-node-selector">kubernetes.taskmanager.node-selector</a></td> + <td>Flink will add additional node selectors specified by the Flink configuration options.</td> + </tr> + <tr> + <td>tolerations</td> + <td>Add the tolerations from <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-tolerations">kubernetes.jobmanager.tolerations</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-tolerations">kubernetes.taskmanager.tolerations</a></td> + <td>Flink will add additional tolerations specified by the Flink configuration options.</td> + </tr> + <tr> + <td>restartPolicy</td> + <td>"always" for JobManager deployment + <br> + "never" for TaskManager pod</td> + <td>The JobManager pod will always be restarted by deployment. And the TaskManager pod should not be restarted.</td> + </tr> + <tr> + <td>serviceAccount</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-service-account">kubernetes.service-account</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-service-account">kubernetes.jobmanager.service-account</a> for JobManager, + <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-service-account">kubernetes.taskmanager.service-account</a> for TaskManager</td> + <td>The JobManager deployment and TaskManager pods will be created with the specified service account.</td> + </tr> + <tr> + <td>volumes</td> + <td>See description</td> + <td>Flink will add some additional internal ConfigMap volumes(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td> + </tr> + </tbody> +</table> + +#### Container Spec +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Container Spec Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>env</td> + <td>Add environment variables from <a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.master.env.{ENV_NAME}</a> + or <a href="{{< ref "docs/deployment/config" >}}#forwarding-environment-variables">containerized.taskmanager.env.{ENV_NAME}</a></td> + <td>Flink will add additional environment variables specified by the Flink configuration options.</td> + </tr> + <tr> + <td>image</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image">kubernetes.container.image</a></td> + <td>The image will be overwritten by flink configuration option.</td> + </tr> + <tr> + <td>imagePullPolicy</td> + <td>Value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-container-image-pull-policy">kubernetes.container.image.pull-policy</a></td> + <td>The image pull policy will be overwritten by flink configuration option.</td> + </tr> + <tr> + <td>name</td> + <td>"flink-job-manager" or "flink-task-manager"</td> + <td>The container name will be overwritten by flink ("flink-job-manager" for the JobManager container, and "flink-task-manager" for every TaskManager container).</td> + </tr> + <tr> + <td>resources</td> + <td>Memory: value of <a href="{{< ref "docs/deployment/config" >}}#jobmanager-memory-process-size">jobmanager.memory.process.size</a> + or <a href="{{< ref "docs/deployment/config" >}}#taskmanager-memory-process-size">taskmanager.memory.process.size</a> + <br> + CPU: value of <a href="{{< ref "docs/deployment/config" >}}#kubernetes-jobmanager-cpu">kubernetes.jobmanager.cpu</a> + or <a href="{{< ref "docs/deployment/config" >}}#kubernetes-taskmanager-cpu">kubernetes.taskmanager.cpu</a></td> + <td>The memory and cpu resources(including requests and limits) will be overwritten by flink. All other resources(e.g. ephemeral-storage) will be retained.</td> + </tr> + <tr> + <td>volumeMounts</td> + <td>See description</td> + <td>Flink will add some additional internal volume mounts(e.g. flink-config-volume, hadoop-config-volume) which is necessary for shipping the Flink configuration and hadoop configuration.</td> + </tr> + </tbody> +</table> + +#### Pod Template Definition Review comment: ```suggestion #### Example of Pod Template ``` ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. Review comment: ```suggestion can be overwritten by Flink. ``` ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template +will always be overridden by Flink. +Refer to the following tables for the full list of pod fields that will be overwritten. +All the fields defined in the pod template and not listed in the tables will be unaffected. + +Please note that some new config options may be introduced to overwrite more Kubernetes fields, +your pod template may not work in the future Flink releases. +We could guarantee that `annotations`, `labels`, `imagePullSecrets`, `nodeSelector`, `tolerations`, +`env`, `init container`, `sidecar container`, `volumes` from pod template will always be respected. + +#### Pod Metadata +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 20%">Pod MetaData Key</th> + <th class="text-left" style="width: 30%">Modified Value</th> + <th class="text-left" style="width: 50%">Description</th> Review comment: Looking at the table, it seems the fields Flink overwrite can be categorized as follows. * **Defined by Flink.** Definitions in template won't take effect. Use cannot control the fields * **Defined by Config Option.** Definitions in template won't take effect. User can control the fields via config options. * **Overwritten by Flink.** Definitions in template will take effect, with additional contents from Flink. I would suggest to explain these categories in advance, and make it a column in the tables. Thus, the tables may contain the following columns: key, category, related config option, description. In this way, the first three columns might be more concise and easier to understand, while the last column provides details that may not be necessary but good to know. ########## File path: docs/content.zh/docs/deployment/resource-providers/native_kubernetes.md ########## @@ -324,4 +324,209 @@ $ kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=edit Please refer to the official Kubernetes documentation on [RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for more information. +### Pod Template + +Flink allows users to define the JobManager and TaskManager via pod template file with the advanced features(e.g. volumes, init container, sidecar container, etc.), +which are not supported by Flink [Kubernetes config options]({{< ref "docs/deployment/config" >}}#kubernetes) directly. +Use [kubernetes.pod-template-file]({{< ref "docs/deployment/config" >}}#kubernetes-pod-template-file) +to specify a local file that contains pod definition. It will be used to initialize the +JobManager and TaskManager instead of an empty pod. Please note that some fields of the pod template Review comment: ```suggestion to specify a local file that contains pod definition. It will be used, instead of an empty pod, to initialize the JobManager and TaskManager. Please note that some fields of the pod template ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
