[jira] [Commented] (FLINK-32561) Change the status field reconciliationTimestamp from long to Date
[ https://issues.apache.org/jira/browse/FLINK-32561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17741871#comment-17741871 ] Xin Hao commented on FLINK-32561: - Agreed~ > Change the status field reconciliationTimestamp from long to Date > - > > Key: FLINK-32561 > URL: https://issues.apache.org/jira/browse/FLINK-32561 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > Can we change the field `status.reconciliationStatus.reconciliationTimestamp` > from long to date? > > At first, this is a broken change for the CRD. > The benefit is that: > # The date format is more human-readable, this is useful when we debug > issues. > # It will be easy to add this field into additionalPrinterColumns with date > duration format. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32561) Change the status field reconciliationTimestamp from long to Date
Xin Hao created FLINK-32561: --- Summary: Change the status field reconciliationTimestamp from long to Date Key: FLINK-32561 URL: https://issues.apache.org/jira/browse/FLINK-32561 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao Can we change the field `status.reconciliationStatus.reconciliationTimestamp` from long to date? At first, this is a broken change for the CRD. The benefit is that: # The date format is more human-readable, this is useful when we debug issues. # It will be easy to add this field into additionalPrinterColumns with date duration format. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-32214) Fetch more info in the FlinkDeployment status by using the overview API
[
https://issues.apache.org/jira/browse/FLINK-32214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-32214.
---
Resolution: Not A Problem
> Fetch more info in the FlinkDeployment status by using the overview API
> ---
>
> Key: FLINK-32214
> URL: https://issues.apache.org/jira/browse/FLINK-32214
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Currently, we are using `/config` API to fetch the FlinkDeployment's basic
> info.
> {code:java}
> {
> "refresh-interval": 3000,
> "timezone-name": "Coordinated Universal Time",
> "timezone-offset": 0,
> "flink-version": "1.15.3",
> "flink-revision": "c41c8e5 @ 2022-11-10T10:39:02+01:00",
> "features": {
> "web-submit": true,
> "web-cancel": false
> }
> } {code}
> Can we switch to using `/overview` to obtain more helpful info?
> {code:java}
> {
> "taskmanagers": 27,
> "slots-total": 27,
> "slots-available": 0,
> "jobs-running": 27,
> "jobs-finished": 0,
> "jobs-cancelled": 2,
> "jobs-failed": 0,
> "flink-version": "1.15.3",
> "flink-commit": "c41c8e5"
> } {code}
> The most useful one for me is `jobs-running`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-32214) Fetch more info in the FlinkDeployment status by using the overview API
[
https://issues.apache.org/jira/browse/FLINK-32214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727365#comment-17727365
]
Xin Hao commented on FLINK-32214:
-
Thanks
> Fetch more info in the FlinkDeployment status by using the overview API
> ---
>
> Key: FLINK-32214
> URL: https://issues.apache.org/jira/browse/FLINK-32214
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Currently, we are using `/config` API to fetch the FlinkDeployment's basic
> info.
> {code:java}
> {
> "refresh-interval": 3000,
> "timezone-name": "Coordinated Universal Time",
> "timezone-offset": 0,
> "flink-version": "1.15.3",
> "flink-revision": "c41c8e5 @ 2022-11-10T10:39:02+01:00",
> "features": {
> "web-submit": true,
> "web-cancel": false
> }
> } {code}
> Can we switch to using `/overview` to obtain more helpful info?
> {code:java}
> {
> "taskmanagers": 27,
> "slots-total": 27,
> "slots-available": 0,
> "jobs-running": 27,
> "jobs-finished": 0,
> "jobs-cancelled": 2,
> "jobs-failed": 0,
> "flink-version": "1.15.3",
> "flink-commit": "c41c8e5"
> } {code}
> The most useful one for me is `jobs-running`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32214) Fetch more info in the FlinkDeployment status by using the overview API
Xin Hao created FLINK-32214:
---
Summary: Fetch more info in the FlinkDeployment status by using
the overview API
Key: FLINK-32214
URL: https://issues.apache.org/jira/browse/FLINK-32214
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
Currently, we are using `/config` API to fetch the FlinkDeployment's basic info.
{code:java}
{
"refresh-interval": 3000,
"timezone-name": "Coordinated Universal Time",
"timezone-offset": 0,
"flink-version": "1.15.3",
"flink-revision": "c41c8e5 @ 2022-11-10T10:39:02+01:00",
"features": {
"web-submit": true,
"web-cancel": false
}
} {code}
Can we switch to using `/overview` to obtain more helpful info?
{code:java}
{
"taskmanagers": 27,
"slots-total": 27,
"slots-available": 0,
"jobs-running": 27,
"jobs-finished": 0,
"jobs-cancelled": 2,
"jobs-failed": 0,
"flink-version": "1.15.3",
"flink-commit": "c41c8e5"
} {code}
The most useful one for me is `jobs-running`.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-30264) Set session job status to FAILED if already have retried max attempts
[ https://issues.apache.org/jira/browse/FLINK-30264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao closed FLINK-30264. --- Resolution: Fixed > Set session job status to FAILED if already have retried max attempts > - > > Key: FLINK-30264 > URL: https://issues.apache.org/jira/browse/FLINK-30264 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > Sometimes, the session job deployment fails because of the user code. > There is nothing the Flink operator can do to fix the failure. > > So can we add a new reconciliation state *FAILED* and set the status to this > if the failure still exists after we have retried for the max attempts? > The reconciliation status will continuously be *UPGRADING* currently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-29566) Reschedule the cleanup logic if cancel job failed
[
https://issues.apache.org/jira/browse/FLINK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17688397#comment-17688397
]
Xin Hao commented on FLINK-29566:
-
I did the required change, but need some time to add the unit tests.
Will finish the unit tests if I can get some free time this week. :)
> Reschedule the cleanup logic if cancel job failed
> -
>
> Key: FLINK-29566
> URL: https://issues.apache.org/jira/browse/FLINK-29566
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Assignee: Shipeng Xie
>Priority: Minor
> Labels: starter
>
> Currently, when we remove the FlinkSessionJob object,
> we always remove the object even if the Flink job is not being canceled
> successfully.
>
> This is *not semantic consistent* if the FlinkSessionJob has been removed but
> the Flink job is still running.
>
> One of the scenarios is that if we deploy a FlinkDeployment with HA mode.
> When we remove the FlinkSessionJob and change the FlinkDeployment at the same
> time,
> or if the TMs are restarting because of some bugs such as OOM.
> Both of these will cause the cancelation of the Flink job to fail because the
> TMs are not available.
>
> We should *reschedule* the cleanup logic if the FlinkDeployment is present.
> And we can add a new ReconciliationState DELETING to indicate the
> FlinkSessionJob's status.
>
> The logic will be
> {code:java}
> if the FlinkDeployment is not present
> delete the FlinkSessionJob object
> else
> if the JM is not available
> reschedule
> else
> if cancel job successfully
> delete the FlinkSessionJob object
> else
> reschedule{code}
> When we cancel the Flink job, we need to verify all the jobs with the same
> name have been deleted in case of the job id is changed after JM restarted.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-30264) Set session job status to FAILED if already have retried max attempts
Xin Hao created FLINK-30264: --- Summary: Set session job status to FAILED if already have retried max attempts Key: FLINK-30264 URL: https://issues.apache.org/jira/browse/FLINK-30264 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao Sometimes, the session job deployment fails because of the user code. There is nothing the Flink operator can do to fix the failure. So can we add a new reconciliation state *FAILED* and set the status to this if the failure still exists after we have retried for the max attempts? The reconciliation status will continuously be *UPGRADING* currently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-29566) Reschedule the cleanup logic if cancel job failed
[
https://issues.apache.org/jira/browse/FLINK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17618264#comment-17618264
]
Xin Hao commented on FLINK-29566:
-
Will submit a PR for that.
> Reschedule the cleanup logic if cancel job failed
> -
>
> Key: FLINK-29566
> URL: https://issues.apache.org/jira/browse/FLINK-29566
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Currently, when we remove the FlinkSessionJob object,
> we always remove the object even if the Flink job is not being canceled
> successfully.
>
> This is *not semantic consistent* if the FlinkSessionJob has been removed but
> the Flink job is still running.
>
> One of the scenarios is that if we deploy a FlinkDeployment with HA mode.
> When we remove the FlinkSessionJob and change the FlinkDeployment at the same
> time,
> or if the TMs are restarting because of some bugs such as OOM.
> Both of these will cause the cancelation of the Flink job to fail because the
> TMs are not available.
>
> We should *reschedule* the cleanup logic if the FlinkDeployment is present.
> And we can add a new ReconciliationState DELETING to indicate the
> FlinkSessionJob's status.
>
> The logic will be
> {code:java}
> if the FlinkDeployment is not present
> delete the FlinkSessionJob object
> else
> if the JM is not available
> reschedule
> else
> if cancel job successfully
> delete the FlinkSessionJob object
> else
> reschedule{code}
> When we cancel the Flink job, we need to verify all the jobs with the same
> name have been deleted in case of the job id is changed after JM restarted.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29581) Emit warning events for session job reconciliation exception
[ https://issues.apache.org/jira/browse/FLINK-29581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-29581: External issue URL: https://github.com/apache/flink-kubernetes-operator/pull/399 > Emit warning events for session job reconciliation exception > > > Key: FLINK-29581 > URL: https://issues.apache.org/jira/browse/FLINK-29581 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > Labels: pull-request-available > > Same as FlinkDeployment, will be useful for monitoring. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29581) Emit warning events for session job reconciliation exception
Xin Hao created FLINK-29581: --- Summary: Emit warning events for session job reconciliation exception Key: FLINK-29581 URL: https://issues.apache.org/jira/browse/FLINK-29581 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao Same as FlinkDeployment, will be useful for monitoring. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-29566) Reschedule the cleanup logic if cancel job failed
[
https://issues.apache.org/jira/browse/FLINK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29566:
Description:
Currently, when we remove the FlinkSessionJob object,
we always remove the object even if the Flink job is not being canceled
successfully.
This is *not semantic consistent* if the FlinkSessionJob has been removed but
the Flink job is still running.
One of the scenarios is that if we deploy a FlinkDeployment with HA mode.
When we remove the FlinkSessionJob and change the FlinkDeployment at the same
time,
or if the TMs are restarting because of some bugs such as OOM.
Both of these will cause the cancelation of the Flink job to fail because the
TMs are not available.
We should *reschedule* the cleanup logic if the FlinkDeployment is present.
And we can add a new ReconciliationState DELETING to indicate the
FlinkSessionJob's status.
The logic will be
{code:java}
if the FlinkDeployment is not present
delete the FlinkSessionJob object
else
if the JM is not available
reschedule
else
if cancel job successfully
delete the FlinkSessionJob object
else
reschedule{code}
When we cancel the Flink job, we need to verify all the jobs with the same name
have been deleted in case of the job id is changed after JM restarted.
was:
Currently, when we remove the FlinkSessionJob object,
we always remove the object even if the Flink job is not being canceled
successfully.
This is not semantic consistent if the FlinkSessionJob has been removed but the
Flink job is still running.
One of the scenarios is that if we deploy a FlinkDeployment with HA mode.
When we remove the FlinkSessionJob and change the FlinkDeployment at the same
time,
or if the TMs are restarting because of some bugs such as OOM.
Both of these will cause the cancelation of the Flink job to fail because the
TMs are not available.
We should reschedule the cleanup logic if the FlinkDeployment is present.
And we can add a new ReconciliationState DELETING to indicate the
FlinkSessionJob's status.
> Reschedule the cleanup logic if cancel job failed
> -
>
> Key: FLINK-29566
> URL: https://issues.apache.org/jira/browse/FLINK-29566
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Currently, when we remove the FlinkSessionJob object,
> we always remove the object even if the Flink job is not being canceled
> successfully.
>
> This is *not semantic consistent* if the FlinkSessionJob has been removed but
> the Flink job is still running.
>
> One of the scenarios is that if we deploy a FlinkDeployment with HA mode.
> When we remove the FlinkSessionJob and change the FlinkDeployment at the same
> time,
> or if the TMs are restarting because of some bugs such as OOM.
> Both of these will cause the cancelation of the Flink job to fail because the
> TMs are not available.
>
> We should *reschedule* the cleanup logic if the FlinkDeployment is present.
> And we can add a new ReconciliationState DELETING to indicate the
> FlinkSessionJob's status.
>
> The logic will be
> {code:java}
> if the FlinkDeployment is not present
> delete the FlinkSessionJob object
> else
> if the JM is not available
> reschedule
> else
> if cancel job successfully
> delete the FlinkSessionJob object
> else
> reschedule{code}
> When we cancel the Flink job, we need to verify all the jobs with the same
> name have been deleted in case of the job id is changed after JM restarted.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29566) Reschedule the cleanup logic if cancel job failed
Xin Hao created FLINK-29566: --- Summary: Reschedule the cleanup logic if cancel job failed Key: FLINK-29566 URL: https://issues.apache.org/jira/browse/FLINK-29566 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao Currently, when we remove the FlinkSessionJob object, we always remove the object even if the Flink job is not being canceled successfully. This is not semantic consistent if the FlinkSessionJob has been removed but the Flink job is still running. One of the scenarios is that if we deploy a FlinkDeployment with HA mode. When we remove the FlinkSessionJob and change the FlinkDeployment at the same time, or if the TMs are restarting because of some bugs such as OOM. Both of these will cause the cancelation of the Flink job to fail because the TMs are not available. We should reschedule the cleanup logic if the FlinkDeployment is present. And we can add a new ReconciliationState DELETING to indicate the FlinkSessionJob's status. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-29561) Log the job id when clean up session job failed
[ https://issues.apache.org/jira/browse/FLINK-29561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-29561: External issue URL: https://github.com/apache/flink-kubernetes-operator/pull/398 > Log the job id when clean up session job failed > --- > > Key: FLINK-29561 > URL: https://issues.apache.org/jira/browse/FLINK-29561 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > If so, we can delete it by Flink rest API manually, and no need to query it > by name. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-29561) Log the job id when clean up session job failed
Xin Hao created FLINK-29561: --- Summary: Log the job id when clean up session job failed Key: FLINK-29561 URL: https://issues.apache.org/jira/browse/FLINK-29561 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao If so, we can delete it by Flink rest API manually, and no need to query it by name. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-29465.
---
Resolution: Not A Problem
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> ...
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610924#comment-17610924
]
Xin Hao commented on FLINK-29465:
-
yes, make sense, will give a try
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> ...
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610895#comment-17610895
]
Xin Hao commented on FLINK-29465:
-
This is useful for decreasing cloud costs. :)
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> ...
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610891#comment-17610891
]
Xin Hao edited comment on FLINK-29465 at 9/29/22 8:42 AM:
--
We currently set the pod resources request and limit the same base on the
resource definition.
For example, if we add resource definition in the FlinkDeployment CR
{code:java}
taskManager:
resource:
memory: 2048m
cpu: 0.5 {code}
The TM's pod definition will be
{code:java}
resources:
limits:
cpu: 500m
memory: 2Gi
requests:
cpu: 500m
memory: 2Gi {code}
According to the Flink KubernetesConfigOptions docs. (For example TM)
{code:java}
TASK_MANAGER_MEMORY_LIMIT_FACTOR =
ConfigOptions.key("kubernetes.taskmanager.memory.limit-factor").doubleType().defaultValue(1.0).withDescription("The
limit factor of memory used by task manager. The resources limit memory will
be set to memory * limit-factor."); {code}
With the limit factor, we can do this
{code:java}
taskManager:
resource:
memory: 512m
memoryLimitFactor: 4
cpu: 0.1
cpuLimitFactor: 2{code}
so that TM's pod spec will be
{code:java}
resources:
limits:
cpu: 100m
memory: 512Mi
requests:
cpu: 200m
memory: 2Gi{code}
was (Author: JIRAUSER286449):
We currently set the pod resources request and limit the same base on the
resource definition.
For example, if we add resource definition in the FlinkDeployment CR
{code:java}
taskManager:
resource:
memory: 2048m
cpu: 0.5 {code}
The TM's pod definition will be
{code:java}
resources:
limits:
cpu: 500m
memory: 2Gi
requests:
cpu: 500m
memory: 2Gi {code}
According to the Flink KubernetesConfigOptions docs. (For example TM)
{code:java}
TASK_MANAGER_MEMORY_LIMIT_FACTOR =
ConfigOptions.key("kubernetes.taskmanager.memory.limit-factor").doubleType().defaultValue(1.0).withDescription("The
limit factor of memory used by task manager. The resources limit memory will
be set to memory * limit-factor."); {code}
With the limit factor, we can do this
{code:java}
taskManager:
resource:
memory: 512m
memoryLimitFactor: 4
cpu: 0.1
cpuLimitFactor: 2{code}
so that TM's pod spec should be
{code:java}
resources:
limits:
cpu: 100m
memory: 512Mi
requests:
cpu: 200m
memory: 2Gi{code}
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> ...
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610891#comment-17610891
]
Xin Hao commented on FLINK-29465:
-
We currently set the pod resources request and limit the same base on the
resource definition.
For example, if we add resource definition in the FlinkDeployment CR
{code:java}
taskManager:
resource:
memory: 2048m
cpu: 0.5 {code}
The TM's pod definition will be
{code:java}
resources:
limits:
cpu: 500m
memory: 2Gi
requests:
cpu: 500m
memory: 2Gi {code}
According to the Flink KubernetesConfigOptions docs. (For example TM)
{code:java}
TASK_MANAGER_MEMORY_LIMIT_FACTOR =
ConfigOptions.key("kubernetes.taskmanager.memory.limit-factor").doubleType().defaultValue(1.0).withDescription("The
limit factor of memory used by task manager. The resources limit memory will
be set to memory * limit-factor."); {code}
With the limit factor, we can do this
{code:java}
taskManager:
resource:
memory: 512m
memoryLimitFactor: 4
cpu: 0.1
cpuLimitFactor: 2{code}
so that TM's pod spec should be
{code:java}
resources:
limits:
cpu: 100m
memory: 512Mi
requests:
cpu: 200m
memory: 2Gi{code}
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> ...
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29465:
Description:
Can we add limit factor supports in the resource definition?
{code:java}
public class Resource {
/** Amount of CPU allocated to the pod. */
private Double cpu;
private double cpuLimitFactor = 1.0;
/** Amount of memory allocated to the pod. Example: 1024m, 1g */
private String memory;
private double memoryLimitFactor = 1.0;
...
} {code}
We can set the defaults as `1.0`, and update them to K8s config options
{code:java}
JOB_MANAGER_CPU_LIMIT_FACTOR
JOB_MANAGER_MEMORY_LIMIT_FACTOR
TASK_MANAGER_CPU_LIMIT_FACTOR
TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
was:
Can we add limit factor supports in the resource definition?
{code:java}
public class Resource {
/** Amount of CPU allocated to the pod. */
private Double cpu;
private double cpuLimitFactor = 1.0;
/** Amount of memory allocated to the pod. Example: 1024m, 1g */
private String memory;
private double memoryLimitFactor = 1.0;
} {code}
We can set the defaults as `1.0`, and update them to K8s config options
{code:java}
JOB_MANAGER_CPU_LIMIT_FACTOR
JOB_MANAGER_MEMORY_LIMIT_FACTOR
TASK_MANAGER_CPU_LIMIT_FACTOR
TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> ...
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29465) Supports limit factor in the resource definition
[
https://issues.apache.org/jira/browse/FLINK-29465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29465:
Description:
Can we add limit factor supports in the resource definition?
{code:java}
public class Resource {
/** Amount of CPU allocated to the pod. */
private Double cpu;
private double cpuLimitFactor = 1.0;
/** Amount of memory allocated to the pod. Example: 1024m, 1g */
private String memory;
private double memoryLimitFactor = 1.0;
} {code}
We can set the defaults as `1.0`, and update them to K8s config options
{code:java}
JOB_MANAGER_CPU_LIMIT_FACTOR
JOB_MANAGER_MEMORY_LIMIT_FACTOR
TASK_MANAGER_CPU_LIMIT_FACTOR
TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
was:
Can we add limit factor supports in the resource definition?
{code:java}
public class Resource {
/** Amount of CPU allocated to the pod. */
private Double cpu;
private double cpuLimitFactor;
/** Amount of memory allocated to the pod. Example: 1024m, 1g */
private String memory;
private double memoryLimitFactor;
} {code}
We can set the defaults as `1.0`, and update them to K8s config options
{code:java}
JOB_MANAGER_CPU_LIMIT_FACTOR
JOB_MANAGER_MEMORY_LIMIT_FACTOR
TASK_MANAGER_CPU_LIMIT_FACTOR
TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
> Supports limit factor in the resource definition
>
>
> Key: FLINK-29465
> URL: https://issues.apache.org/jira/browse/FLINK-29465
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add limit factor supports in the resource definition?
> {code:java}
> public class Resource {
> /** Amount of CPU allocated to the pod. */
> private Double cpu;
> private double cpuLimitFactor = 1.0;
> /** Amount of memory allocated to the pod. Example: 1024m, 1g */
> private String memory;
> private double memoryLimitFactor = 1.0;
> } {code}
> We can set the defaults as `1.0`, and update them to K8s config options
> {code:java}
> JOB_MANAGER_CPU_LIMIT_FACTOR
> JOB_MANAGER_MEMORY_LIMIT_FACTOR
> TASK_MANAGER_CPU_LIMIT_FACTOR
> TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29465) Supports limit factor in the resource definition
Xin Hao created FLINK-29465:
---
Summary: Supports limit factor in the resource definition
Key: FLINK-29465
URL: https://issues.apache.org/jira/browse/FLINK-29465
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
Can we add limit factor supports in the resource definition?
{code:java}
public class Resource {
/** Amount of CPU allocated to the pod. */
private Double cpu;
private double cpuLimitFactor;
/** Amount of memory allocated to the pod. Example: 1024m, 1g */
private String memory;
private double memoryLimitFactor;
} {code}
We can set the defaults as `1.0`, and update them to K8s config options
{code:java}
JOB_MANAGER_CPU_LIMIT_FACTOR
JOB_MANAGER_MEMORY_LIMIT_FACTOR
TASK_MANAGER_CPU_LIMIT_FACTOR
TASK_MANAGER_MEMORY_LIMIT_FACTOR {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29383) Add additionalPrinterColumns definition (PrinterColumn annotation) for some status fields
[
https://issues.apache.org/jira/browse/FLINK-29383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29383:
Summary: Add additionalPrinterColumns definition (PrinterColumn annotation)
for some status fields (was: Add additionalPrinterColumns definition for some
status fields)
> Add additionalPrinterColumns definition (PrinterColumn annotation) for some
> status fields
> -
>
> Key: FLINK-29383
> URL: https://issues.apache.org/jira/browse/FLINK-29383
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> We should add additionalPrinterColumns definitions in the CRD so that we can
> use
> {code:java}
> k get flinksessionjob -o wide
> {code}
> to see the session jobs statuses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29383) Add additionalPrinterColumns definition for some status fields
[
https://issues.apache.org/jira/browse/FLINK-29383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29383:
Summary: Add additionalPrinterColumns definition for some status fields
(was: Add additionalPrinterColumns for some status fields)
> Add additionalPrinterColumns definition for some status fields
> --
>
> Key: FLINK-29383
> URL: https://issues.apache.org/jira/browse/FLINK-29383
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> We should add additionalPrinterColumns definitions in the CRD so that we can
> use
> {code:java}
> k get flinksessionjob -o wide
> {code}
> to see the session jobs statuses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29383) Add additionalPrinterColumns for some status fields
Xin Hao created FLINK-29383:
---
Summary: Add additionalPrinterColumns for some status fields
Key: FLINK-29383
URL: https://issues.apache.org/jira/browse/FLINK-29383
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
We should add additionalPrinterColumns definitions in the CRD so that we can use
{code:java}
k get flinksessionjob -o wide
{code}
to see the session jobs statuses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29361) How to set headers with the new Flink KafkaSink
[
https://issues.apache.org/jira/browse/FLINK-29361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607585#comment-17607585
]
Xin Hao commented on FLINK-29361:
-
thank you guys, will give a try~
> How to set headers with the new Flink KafkaSink
> ---
>
> Key: FLINK-29361
> URL: https://issues.apache.org/jira/browse/FLINK-29361
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
> that it's not possible to add Kafka record headers.
> I think we should add this feature or document it if we already have it.
>
> Below code is what we can do with FlinkKafkaProducer and ProducerRecord
> {code:java}
> public class SomeKafkaSerializationSchema
> implements KafkaSerializationSchema {
> ...
> @Override public ProducerRecord serialize(T t, Long ts) {
> ...
> var record = ProducerRecord(topic, some_bytes_a);
> record.headers().add("id", some_bytes_b);
> return record;
> }}
> ...
> var producer = new FlinkKafkaProducer<>(
> topic,
> new SomeKafkaSerializationSchema<>(...),
> producerProps,
> FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
> );
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-29361) How to set headers with the new Flink KafkaSink
[
https://issues.apache.org/jira/browse/FLINK-29361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-29361.
---
Resolution: Not A Problem
> How to set headers with the new Flink KafkaSink
> ---
>
> Key: FLINK-29361
> URL: https://issues.apache.org/jira/browse/FLINK-29361
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
> that it's not possible to add Kafka record headers.
> I think we should add this feature or document it if we already have it.
>
> Below code is what we can do with FlinkKafkaProducer and ProducerRecord
> {code:java}
> public class SomeKafkaSerializationSchema
> implements KafkaSerializationSchema {
> ...
> @Override public ProducerRecord serialize(T t, Long ts) {
> ...
> var record = ProducerRecord(topic, some_bytes_a);
> record.headers().add("id", some_bytes_b);
> return record;
> }}
> ...
> var producer = new FlinkKafkaProducer<>(
> topic,
> new SomeKafkaSerializationSchema<>(...),
> producerProps,
> FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
> );
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-29361) How to set headers with the new Flink KafkaSink
[
https://issues.apache.org/jira/browse/FLINK-29361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607559#comment-17607559
]
Xin Hao commented on FLINK-29361:
-
Added
> How to set headers with the new Flink KafkaSink
> ---
>
> Key: FLINK-29361
> URL: https://issues.apache.org/jira/browse/FLINK-29361
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
> that it's not possible to add Kafka record headers.
> I think we should add this feature or document it if we already have it.
>
> Below code is what we can do with FlinkKafkaProducer and ProducerRecord
> {code:java}
> public class SomeKafkaSerializationSchema
> implements KafkaSerializationSchema {
> ...
> @Override public ProducerRecord serialize(T t, Long ts) {
> ...
> var record = ProducerRecord(topic, some_bytes_a);
> record.headers().add("id", some_bytes_b);
> return record;
> }}
> ...
> var producer = new FlinkKafkaProducer<>(
> topic,
> new SomeKafkaSerializationSchema<>(...),
> producerProps,
> FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
> );
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29361) How to set headers with the new Flink KafkaSink
[
https://issues.apache.org/jira/browse/FLINK-29361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29361:
Description:
I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
that it's not possible to add Kafka record headers.
I think we should add this feature or document it if we already have it.
Below code is what we can do with FlinkKafkaProducer and ProducerRecord
{code:java}
public class SomeKafkaSerializationSchema
implements KafkaSerializationSchema {
...
@Override public ProducerRecord serialize(T t, Long ts) {
...
var record = ProducerRecord(topic, some_bytes_a);
record.headers().add("id", some_bytes_b);
return record;
}}
...
var producer = new FlinkKafkaProducer<>(
topic,
new SomeKafkaSerializationSchema<>(...),
producerProps,
FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
);
{code}
was:
I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
that it's not possible to add Kafka record headers.
I think we should add this feature or document it if we already have it.
Below code is what we can do with ProducerRecord
{code:java}
public class SomeKafkaSerializationSchema
implements KafkaSerializationSchema {
...
@Override public ProducerRecord serialize(T t, Long ts) {
...
var record = ProducerRecord(topic, some_bytes_a);
record.headers().add("id", some_bytes_b);
return record;
}}
...
var producer = new FlinkKafkaProducer<>(
topic,
new SomeKafkaSerializationSchema<>(...),
producerProps,
FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
);
{code}
> How to set headers with the new Flink KafkaSink
> ---
>
> Key: FLINK-29361
> URL: https://issues.apache.org/jira/browse/FLINK-29361
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
> that it's not possible to add Kafka record headers.
> I think we should add this feature or document it if we already have it.
>
> Below code is what we can do with FlinkKafkaProducer and ProducerRecord
> {code:java}
> public class SomeKafkaSerializationSchema
> implements KafkaSerializationSchema {
> ...
> @Override public ProducerRecord serialize(T t, Long ts) {
> ...
> var record = ProducerRecord(topic, some_bytes_a);
> record.headers().add("id", some_bytes_b);
> return record;
> }}
> ...
> var producer = new FlinkKafkaProducer<>(
> topic,
> new SomeKafkaSerializationSchema<>(...),
> producerProps,
> FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
> );
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-29361) How to set headers with the new Flink KafkaSink
[
https://issues.apache.org/jira/browse/FLINK-29361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-29361:
Description:
I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
that it's not possible to add Kafka record headers.
I think we should add this feature or document it if we already have it.
Below code is what we can do with ProducerRecord
{code:java}
public class SomeKafkaSerializationSchema
implements KafkaSerializationSchema {
...
@Override public ProducerRecord serialize(T t, Long ts) {
...
var record = ProducerRecord(topic, some_bytes_a);
record.headers().add("id", some_bytes_b);
return record;
}}
...
var producer = new FlinkKafkaProducer<>(
topic,
new SomeKafkaSerializationSchema<>(...),
producerProps,
FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
);
{code}
was:
I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
that it's not possible to add Kafka record headers.
I think we should add this feature or document it if we already have it.
> How to set headers with the new Flink KafkaSink
> ---
>
> Key: FLINK-29361
> URL: https://issues.apache.org/jira/browse/FLINK-29361
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems
> that it's not possible to add Kafka record headers.
> I think we should add this feature or document it if we already have it.
>
> Below code is what we can do with ProducerRecord
>
> {code:java}
> public class SomeKafkaSerializationSchema
> implements KafkaSerializationSchema {
> ...
> @Override public ProducerRecord serialize(T t, Long ts) {
> ...
> var record = ProducerRecord(topic, some_bytes_a);
> record.headers().add("id", some_bytes_b);
> return record;
> }}
> ...
> var producer = new FlinkKafkaProducer<>(
> topic,
> new SomeKafkaSerializationSchema<>(...),
> producerProps,
> FlinkKafkaProducer.Semantic.AT_LEAST_ONCE
> );
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29361) How to set headers with the new Flink KafkaSink
Xin Hao created FLINK-29361: --- Summary: How to set headers with the new Flink KafkaSink Key: FLINK-29361 URL: https://issues.apache.org/jira/browse/FLINK-29361 Project: Flink Issue Type: Improvement Components: Connectors / Kafka Reporter: Xin Hao I'm using Flink 1.15.2, when I try to migrate to the new KafkaSink, it seems that it's not possible to add Kafka record headers. I think we should add this feature or document it if we already have it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585233#comment-17585233
]
Xin Hao edited comment on FLINK-28979 at 8/26/22 8:30 AM:
--
The reason is that I will see the linked resources in the Argo CD's Web UI.
Not something important, but will be a bit more user-friendly.
!image-2022-08-26-16-27-59-389.png|width=463,height=127!
was (Author: JIRAUSER286449):
The reason is that I can see the linked resources in the Argo CD's Web UI.
Not something important, but will be a bit more user-friendly.
!image-2022-08-26-16-27-59-389.png|width=463,height=127!
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: image-2022-08-26-16-27-59-389.png
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585233#comment-17585233
]
Xin Hao commented on FLINK-28979:
-
The reason is that I can see the linked resources in the Argo CD's Web UI.
Not something important, but will be a bit more user-friendly.
!image-2022-08-26-16-27-59-389.png|width=463,height=127!
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: image-2022-08-26-16-27-59-389.png
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28979:
Attachment: image-2022-08-26-16-27-59-389.png
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: image-2022-08-26-16-27-59-389.png
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28979:
Attachment: (was: 111)
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (FLINK-28979) Add another owner into the JM deployment's owner references
[ https://issues.apache.org/jira/browse/FLINK-28979 ]
Xin Hao deleted comment on FLINK-28979:
-
was (Author: JIRAUSER286449):
The reason is that I will see the linked UI from Argo CD UI with the owner
reference.
!111.jpg!
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585228#comment-17585228
]
Xin Hao commented on FLINK-28979:
-
The reason is that I will see the linked UI from Argo CD UI with the owner
reference.
!111.jpg!
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111, 111.jpg
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28979:
Attachment: (was: 111.jpg)
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28979:
Attachment: 111.jpg
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (FLINK-28979) Add another owner into the JM deployment's owner references
[ https://issues.apache.org/jira/browse/FLINK-28979 ]
Xin Hao deleted comment on FLINK-28979:
-
was (Author: JIRAUSER286449):
The reason is that I will see the linked UI from Argo CD UI with the owner
reference.
[^111]
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585223#comment-17585223
]
Xin Hao commented on FLINK-28979:
-
The reason is that I will see the linked UI from Argo CD UI with the owner
reference.
[^111]
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28979:
Attachment: 111
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
> Attachments: 111
>
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-29086) Fix the Helm chart's Pod env reference
Xin Hao created FLINK-29086: --- Summary: Fix the Helm chart's Pod env reference Key: FLINK-29086 URL: https://issues.apache.org/jira/browse/FLINK-29086 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao We need to add a `quote` pipeline to the env params reference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-28979) Add another owner into the JM deployment's owner references
[
https://issues.apache.org/jira/browse/FLINK-28979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28979:
Description:
This action will link the resources into the FlinkDeployment
The JM deployment will look like
{code:java}
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: Deployment
name: xxx
- apiVersion: flink.apache.org/v1beta1
blockOwnerDeletion: true
controller: false
kind: FlinkDeployment
name: xxx{code}
was:
This action will link the resources into the FlinkDeployment
Will look like
{code:java}
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: Deployment
name: xxx
- apiVersion: flink.apache.org/v1beta1
blockOwnerDeletion: true
controller: false
kind: FlinkDeployment
name: xxx{code}
> Add another owner into the JM deployment's owner references
> ---
>
> Key: FLINK-28979
> URL: https://issues.apache.org/jira/browse/FLINK-28979
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> This action will link the resources into the FlinkDeployment
> The JM deployment will look like
> {code:java}
> ownerReferences:
> - apiVersion: apps/v1
> blockOwnerDeletion: true
> controller: true
> kind: Deployment
> name: xxx
> - apiVersion: flink.apache.org/v1beta1
> blockOwnerDeletion: true
> controller: false
> kind: FlinkDeployment
> name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-28979) Add another owner into the JM deployment's owner references
Xin Hao created FLINK-28979:
---
Summary: Add another owner into the JM deployment's owner
references
Key: FLINK-28979
URL: https://issues.apache.org/jira/browse/FLINK-28979
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
This action will link the resources into the FlinkDeployment
Will look like
{code:java}
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: Deployment
name: xxx
- apiVersion: flink.apache.org/v1beta1
blockOwnerDeletion: true
controller: false
kind: FlinkDeployment
name: xxx{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-28615) Add K8s recommend labels to the JM deployments, JM services created by operator
[
https://issues.apache.org/jira/browse/FLINK-28615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-28615.
---
Resolution: Duplicate
> Add K8s recommend labels to the JM deployments, JM services created by
> operator
> ---
>
> Key: FLINK-28615
> URL: https://issues.apache.org/jira/browse/FLINK-28615
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using the Flink operator with Argo CD, it will be nice if we can add the
> K8s recommend labels to the deployments and services. Such as:
> {code:java}
> labels:
> app.kubernetes.io/managed-by: apache-flink-operator
> app.kubernetes.io/part-of: flink-session-cluster-a {code}
> With this, the users can see all the resources created by FlinkDeployment in
> the Argo CD Web UI
>
> See also:
> [https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28869) Emit warning event for ClusterDeploymentException
[ https://issues.apache.org/jira/browse/FLINK-28869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-28869: External issue URL: https://github.com/apache/flink-kubernetes-operator/pull/341 > Emit warning event for ClusterDeploymentException > - > > Key: FLINK-28869 > URL: https://issues.apache.org/jira/browse/FLINK-28869 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > Labels: pull-request-available > > We should emit a warning event for ClusterDeploymentException also. > We only send the waning events for DeploymentFailedException currently. > I met this when I have some invalid K8s spec (ConfigMap) in the PodTemplate -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-28869) Emit warning event for ClusterDeploymentException
Xin Hao created FLINK-28869: --- Summary: Emit warning event for ClusterDeploymentException Key: FLINK-28869 URL: https://issues.apache.org/jira/browse/FLINK-28869 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao We should emit a warning event for ClusterDeploymentException also. We only send the waning events for DeploymentFailedException currently. I met this when I have some invalid K8s spec (ConfigMap) in the PodTemplate -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-28615) Add K8s recommend labels to the JM deployments, JM services created by operator
[
https://issues.apache.org/jira/browse/FLINK-28615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28615:
Summary: Add K8s recommend labels to the JM deployments, JM services
created by operator (was: Add K8s recommend labels to the deployments,
services created by operator)
> Add K8s recommend labels to the JM deployments, JM services created by
> operator
> ---
>
> Key: FLINK-28615
> URL: https://issues.apache.org/jira/browse/FLINK-28615
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using the Flink operator with Argo CD, it will be nice if we can add the
> K8s recommend labels to the deployments and services. Such as:
> {code:java}
> labels:
> app.kubernetes.io/managed-by: apache-flink-operator
> app.kubernetes.io/part-of: flink-session-cluster-a {code}
> With this, the users can see all the resources created by FlinkDeployment in
> the Argo CD Web UI
>
> See also:
> [https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-28615) Add K8s recommend labels to the deployments, services created by operator
[
https://issues.apache.org/jira/browse/FLINK-28615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570459#comment-17570459
]
Xin Hao commented on FLINK-28615:
-
For the operator itself, it's also better to have these labels ~
> Add K8s recommend labels to the deployments, services created by operator
> -
>
> Key: FLINK-28615
> URL: https://issues.apache.org/jira/browse/FLINK-28615
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using the Flink operator with Argo CD, it will be nice if we can add the
> K8s recommend labels to the deployments and services. Such as:
> {code:java}
> labels:
> app.kubernetes.io/managed-by: apache-flink-operator
> app.kubernetes.io/part-of: flink-session-cluster-a {code}
> With this, the users can see all the resources created by FlinkDeployment in
> the Argo CD Web UI
>
> See also:
> [https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (FLINK-28615) Add K8s recommend labels to the deployments, services created by operator
[
https://issues.apache.org/jira/browse/FLINK-28615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17570458#comment-17570458
]
Xin Hao commented on FLINK-28615:
-
Sorry for the unclear description, I'm saying the Flink cluster deployments,
not the operator itself.
> Add K8s recommend labels to the deployments, services created by operator
> -
>
> Key: FLINK-28615
> URL: https://issues.apache.org/jira/browse/FLINK-28615
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using the Flink operator with Argo CD, it will be nice if we can add the
> K8s recommend labels to the deployments and services. Such as:
> {code:java}
> labels:
> app.kubernetes.io/managed-by: apache-flink-operator
> app.kubernetes.io/part-of: flink-session-cluster-a {code}
> With this, the users can see all the resources created by FlinkDeployment in
> the Argo CD Web UI
>
> See also:
> [https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (FLINK-28615) Add K8s recommend labels to the deployments, services created by operator
[
https://issues.apache.org/jira/browse/FLINK-28615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28615:
Description:
I'm using the Flink operator with Argo CD, it will be nice if we can add the
K8s recommend labels to the deployments and services. Such as:
{code:java}
labels:
app.kubernetes.io/managed-by: apache-flink-operator
app.kubernetes.io/part-of: flink-session-cluster-a {code}
With this, the users can see all the resources created by FlinkDeployment in
the Argo CD Web UI
See also:
[https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels]
was:
I'm using the Flink operator with Argo CD, it will be nice if we can add the
K8s recommend labels to the deployments and services. Such as:
{code:java}
labels:
app.kubernetes.io/managed-by: apache-flink-operator
app.kubernetes.io/part-of: flink-session-cluster-a {code}
With this, the users can see all the resources created by FlinkDeployment
See also:
https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels
> Add K8s recommend labels to the deployments, services created by operator
> -
>
> Key: FLINK-28615
> URL: https://issues.apache.org/jira/browse/FLINK-28615
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I'm using the Flink operator with Argo CD, it will be nice if we can add the
> K8s recommend labels to the deployments and services. Such as:
> {code:java}
> labels:
> app.kubernetes.io/managed-by: apache-flink-operator
> app.kubernetes.io/part-of: flink-session-cluster-a {code}
> With this, the users can see all the resources created by FlinkDeployment in
> the Argo CD Web UI
>
> See also:
> [https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-28615) Add K8s recommend labels to the deployments, services created by operator
Xin Hao created FLINK-28615:
---
Summary: Add K8s recommend labels to the deployments, services
created by operator
Key: FLINK-28615
URL: https://issues.apache.org/jira/browse/FLINK-28615
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
I'm using the Flink operator with Argo CD, it will be nice if we can add the
K8s recommend labels to the deployments and services. Such as:
{code:java}
labels:
app.kubernetes.io/managed-by: apache-flink-operator
app.kubernetes.io/part-of: flink-session-cluster-a {code}
With this, the users can see all the resources created by FlinkDeployment
See also:
https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/#labels
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Closed] (FLINK-28438) We should clear the error message after the session job is running/finished
[ https://issues.apache.org/jira/browse/FLINK-28438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao closed FLINK-28438. --- Resolution: Cannot Reproduce > We should clear the error message after the session job is running/finished > --- > > Key: FLINK-28438 > URL: https://issues.apache.org/jira/browse/FLINK-28438 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Assignee: Xin Hao >Priority: Minor > Labels: pull-request-available > > Once an error occurs, the error filed will contain the message even if the > job is running successfully at the end. > We should clear the error message when the session job is running or finished. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-28438) We should clear the error message after the session job is running/finished
[ https://issues.apache.org/jira/browse/FLINK-28438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-28438: Description: Once an error occurs, the error filed will contain the message even if the job is running successfully at the end. We should clear the error message when the session job is running or finished. was: Once there is an error occurs, the error filed will contain the message even if the job are running successfully. We should clear the error message when the session job is running or finished. > We should clear the error message after the session job is running/finished > --- > > Key: FLINK-28438 > URL: https://issues.apache.org/jira/browse/FLINK-28438 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > Labels: pull-request-available > > Once an error occurs, the error filed will contain the message even if the > job is running successfully at the end. > We should clear the error message when the session job is running or finished. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-28438) We should clear the error message after the session job is running/finished
[ https://issues.apache.org/jira/browse/FLINK-28438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-28438: Description: Once there is an error occurs, the error filed will contain the message even if the job are running successfully. We should clear the error message when the session job is running or finished. > We should clear the error message after the session job is running/finished > --- > > Key: FLINK-28438 > URL: https://issues.apache.org/jira/browse/FLINK-28438 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > Labels: pull-request-available > > Once there is an error occurs, the error filed will contain the message even > if the job are running successfully. > We should clear the error message when the session job is running or finished. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-28438) We should clear the error message after the session job is running/finished
Xin Hao created FLINK-28438: --- Summary: We should clear the error message after the session job is running/finished Key: FLINK-28438 URL: https://issues.apache.org/jira/browse/FLINK-28438 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-28385) Change the validator to return an error if the Jar URI is an empty string
Xin Hao created FLINK-28385: --- Summary: Change the validator to return an error if the Jar URI is an empty string Key: FLINK-28385 URL: https://issues.apache.org/jira/browse/FLINK-28385 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (FLINK-28225) Supports custom ENVs in the Helm chart
[
https://issues.apache.org/jira/browse/FLINK-28225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28225:
External issue URL:
https://github.com/apache/flink-kubernetes-operator/pull/276
> Supports custom ENVs in the Helm chart
> --
>
> Key: FLINK-28225
> URL: https://issues.apache.org/jira/browse/FLINK-28225
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add custom ENVs supports in the operator Helm?
> Such as:
> {code:java}
> # In the values.yaml
> operatorEnvs:
> # - name: ""
> # value: ""
> webhookEnvs:
> # - name: ""
> # value: ""{code}
> {code:java}
> # In the deployment.yaml
> env:
> - name: name1
> value: value1
> {{- range $k, $v := .Values.operatorEnvs }}
> - name: {{ $v.name }}
> value: {{ $v.value }}
> {{- end }}{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-28225) Supports custom ENVs in the Helm chart
[
https://issues.apache.org/jira/browse/FLINK-28225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558008#comment-17558008
]
Xin Hao commented on FLINK-28225:
-
https://github.com/apache/flink-kubernetes-operator/pull/276/files
> Supports custom ENVs in the Helm chart
> --
>
> Key: FLINK-28225
> URL: https://issues.apache.org/jira/browse/FLINK-28225
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add custom ENVs supports in the operator Helm?
> Such as:
> {code:java}
> # In the values.yaml
> operatorEnvs:
> # - name: ""
> # value: ""
> webhookEnvs:
> # - name: ""
> # value: ""{code}
> {code:java}
> # In the deployment.yaml
> env:
> - name: name1
> value: value1
> {{- range $k, $v := .Values.operatorEnvs }}
> - name: {{ $v.name }}
> value: {{ $v.value }}
> {{- end }}{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-28225) Supports custom ENVs in the Helm chart
[
https://issues.apache.org/jira/browse/FLINK-28225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17558004#comment-17558004
]
Xin Hao commented on FLINK-28225:
-
got it
> Supports custom ENVs in the Helm chart
> --
>
> Key: FLINK-28225
> URL: https://issues.apache.org/jira/browse/FLINK-28225
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add custom ENVs supports in the operator Helm?
> Such as:
> {code:java}
> # In the values.yaml
> operatorEnvs:
> # - name: ""
> # value: ""
> webhookEnvs:
> # - name: ""
> # value: ""{code}
> {code:java}
> # In the deployment.yaml
> env:
> - name: name1
> value: value1
> {{- range $k, $v := .Values.operatorEnvs }}
> - name: {{ $v.name }}
> value: {{ $v.value }}
> {{- end }}{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-28225) Supports custom ENVs in the Helm chart
[
https://issues.apache.org/jira/browse/FLINK-28225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao updated FLINK-28225:
Description:
Can we add custom ENVs supports in the operator Helm?
Such as:
{code:java}
# In the values.yaml
operatorEnvs:
# - name: ""
# value: ""
webhookEnvs:
# - name: ""
# value: ""{code}
{code:java}
# In the deployment.yaml
env:
- name: name1
value: value1
{{- range $k, $v := .Values.operatorEnvs }}
- name: {{ $v.name }}
value: {{ $v.value }}
{{- end }}{code}
was:
Can we add custom ENVs supports in the operator Helm?
Such as:
{code:java}
# In the values.yaml
operatorEnvs:
# - name: ""
# value: ""
webhookEnvs:
# - name: ""
# value: ""{code}
{code:java}
# In the deployment.yaml
env:
- name: name1
value: value1
{{- range $k, $v := .Values.operatorEnvs }} {code}
> Supports custom ENVs in the Helm chart
> --
>
> Key: FLINK-28225
> URL: https://issues.apache.org/jira/browse/FLINK-28225
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Can we add custom ENVs supports in the operator Helm?
> Such as:
> {code:java}
> # In the values.yaml
> operatorEnvs:
> # - name: ""
> # value: ""
> webhookEnvs:
> # - name: ""
> # value: ""{code}
> {code:java}
> # In the deployment.yaml
> env:
> - name: name1
> value: value1
> {{- range $k, $v := .Values.operatorEnvs }}
> - name: {{ $v.name }}
> value: {{ $v.value }}
> {{- end }}{code}
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (FLINK-28225) Supports custom ENVs in the Helm chart
Xin Hao created FLINK-28225:
---
Summary: Supports custom ENVs in the Helm chart
Key: FLINK-28225
URL: https://issues.apache.org/jira/browse/FLINK-28225
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
Can we add custom ENVs supports in the operator Helm?
Such as:
{code:java}
# In the values.yaml
operatorEnvs:
# - name: ""
# value: ""
webhookEnvs:
# - name: ""
# value: ""{code}
{code:java}
# In the deployment.yaml
env:
- name: name1
value: value1
{{- range $k, $v := .Values.operatorEnvs }} {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (FLINK-28012) Add built-in support for fetching jar from GCS (Google Cloud Storage)
[
https://issues.apache.org/jira/browse/FLINK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-28012.
---
Resolution: Not A Problem
> Add built-in support for fetching jar from GCS (Google Cloud Storage)
> -
>
> Key: FLINK-28012
> URL: https://issues.apache.org/jira/browse/FLINK-28012
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I think it's easy and meaningful to add built-in support in session job mode
> for downloading jar from GCS now.
> The logic should look like the code below
> {code:java}
> if ("http".equals(uri.getScheme()) || "https".equals(uri.getScheme())) {
> return HttpArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else if ("gs".equals(uri.getScheme())) {
> return GcsArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else {
> return FileSystemBasedArtifactFetcher.INSTANCE.fetch(
> jarURI, flinkConfiguration, targetDir);
> } {code}
> We only need to extend the ArtifactManager and implement the new
> GcsArtifactFetcher.
> Also, the users can set up the GCS authorization credentials by ENV and
> ConfigMap so that no need for any change inner the operator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-28012) Add built-in support for fetching jar from GCS (Google Cloud Storage)
[
https://issues.apache.org/jira/browse/FLINK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553422#comment-17553422
]
Xin Hao commented on FLINK-28012:
-
Thanks a lot!
> Add built-in support for fetching jar from GCS (Google Cloud Storage)
> -
>
> Key: FLINK-28012
> URL: https://issues.apache.org/jira/browse/FLINK-28012
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I think it's easy and meaningful to add built-in support in session job mode
> for downloading jar from GCS now.
> The logic should look like the code below
> {code:java}
> if ("http".equals(uri.getScheme()) || "https".equals(uri.getScheme())) {
> return HttpArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else if ("gs".equals(uri.getScheme())) {
> return GcsArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else {
> return FileSystemBasedArtifactFetcher.INSTANCE.fetch(
> jarURI, flinkConfiguration, targetDir);
> } {code}
> We only need to extend the ArtifactManager and implement the new
> GcsArtifactFetcher.
> Also, the users can set up the GCS authorization credentials by ENV and
> ConfigMap so that no need for any change inner the operator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-28012) Add built-in support for fetching jar from GCS (Google Cloud Storage)
[
https://issues.apache.org/jira/browse/FLINK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553391#comment-17553391
]
Xin Hao commented on FLINK-28012:
-
Haven't tested that but maybe it's great if we can add some instructions for
the Operator just like Flink does.
[https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/gcs/]
> Add built-in support for fetching jar from GCS (Google Cloud Storage)
> -
>
> Key: FLINK-28012
> URL: https://issues.apache.org/jira/browse/FLINK-28012
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I think it's easy and meaningful to add built-in support in session job mode
> for downloading jar from GCS now.
> The logic should look like the code below
> {code:java}
> if ("http".equals(uri.getScheme()) || "https".equals(uri.getScheme())) {
> return HttpArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else if ("gs".equals(uri.getScheme())) {
> return GcsArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else {
> return FileSystemBasedArtifactFetcher.INSTANCE.fetch(
> jarURI, flinkConfiguration, targetDir);
> } {code}
> We only need to extend the ArtifactManager and implement the new
> GcsArtifactFetcher.
> Also, the users can set up the GCS authorization credentials by ENV and
> ConfigMap so that no need for any change inner the operator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-28012) Add built-in support for fetching jar from GCS (Google Cloud Storage)
[
https://issues.apache.org/jira/browse/FLINK-28012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17553252#comment-17553252
]
Xin Hao commented on FLINK-28012:
-
> have you tried by adding the flink-gs-fs-hadoop dependency ?
No, will give it a try later.
> Add built-in support for fetching jar from GCS (Google Cloud Storage)
> -
>
> Key: FLINK-28012
> URL: https://issues.apache.org/jira/browse/FLINK-28012
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> I think it's easy and meaningful to add built-in support in session job mode
> for downloading jar from GCS now.
> The logic should look like the code below
> {code:java}
> if ("http".equals(uri.getScheme()) || "https".equals(uri.getScheme())) {
> return HttpArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else if ("gs".equals(uri.getScheme())) {
> return GcsArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
> targetDir);
> } else {
> return FileSystemBasedArtifactFetcher.INSTANCE.fetch(
> jarURI, flinkConfiguration, targetDir);
> } {code}
> We only need to extend the ArtifactManager and implement the new
> GcsArtifactFetcher.
> Also, the users can set up the GCS authorization credentials by ENV and
> ConfigMap so that no need for any change inner the operator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (FLINK-28012) Add built-in support for fetching jar from GCS (Google Cloud Storage)
Xin Hao created FLINK-28012:
---
Summary: Add built-in support for fetching jar from GCS (Google
Cloud Storage)
Key: FLINK-28012
URL: https://issues.apache.org/jira/browse/FLINK-28012
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
I think it's easy and meaningful to add built-in support in session job mode
for downloading jar from GCS now.
The logic should look like the code below
{code:java}
if ("http".equals(uri.getScheme()) || "https".equals(uri.getScheme())) {
return HttpArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
targetDir);
} else if ("gs".equals(uri.getScheme())) {
return GcsArtifactFetcher.INSTANCE.fetch(jarURI, flinkConfiguration,
targetDir);
} else {
return FileSystemBasedArtifactFetcher.INSTANCE.fetch(
jarURI, flinkConfiguration, targetDir);
} {code}
We only need to extend the ArtifactManager and implement the new
GcsArtifactFetcher.
Also, the users can set up the GCS authorization credentials by ENV and
ConfigMap so that no need for any change inner the operator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Updated] (FLINK-27600) Add basic logs for coping and deleting jars
[ https://issues.apache.org/jira/browse/FLINK-27600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-27600: Description: Because we delete the jar after submitting the job, it's better to leave some logs for the end-users. (was: Because we delete the jar after submitted the job, it's better to leave some logs for the end users.) > Add basic logs for coping and deleting jars > --- > > Key: FLINK-27600 > URL: https://issues.apache.org/jira/browse/FLINK-27600 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > Because we delete the jar after submitting the job, it's better to leave some > logs for the end-users. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (FLINK-27600) Add basic logs for coping and deleting jars
Xin Hao created FLINK-27600: --- Summary: Add basic logs for coping and deleting jars Key: FLINK-27600 URL: https://issues.apache.org/jira/browse/FLINK-27600 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao Because we delete the jar after submitted the job, it's better to leave some logs for the end users. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Created] (FLINK-27595) Helm: We should make security context configurable
Xin Hao created FLINK-27595: --- Summary: Helm: We should make security context configurable Key: FLINK-27595 URL: https://issues.apache.org/jira/browse/FLINK-27595 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao We can extract pod security context to Helm's Values to make it configurable. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Closed] (FLINK-27370) Add a new SessionJobState - Failed
[
https://issues.apache.org/jira/browse/FLINK-27370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-27370.
---
Resolution: Not A Problem
> Add a new SessionJobState - Failed
> --
>
> Key: FLINK-27370
> URL: https://issues.apache.org/jira/browse/FLINK-27370
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> It will be nice if we can add a new SessionJobState Failed to indicate there
> is an error for the session job.
> {code:java}
> status:
> error: 'The error message'
> jobStatus:
> savepointInfo: {}
> reconciliationStatus:
> reconciliationTimestamp: 0
> state: DEPLOYED {code}
> Reason:
> 1. It will be easier for monitoring
> 2. I have a personal controller to submit session jobs, it will be cleaner to
> check the state by a single field and get the details by the error field.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-27370) Add a new SessionJobState - Failed
[
https://issues.apache.org/jira/browse/FLINK-27370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528500#comment-17528500
]
Xin Hao commented on FLINK-27370:
-
k, got it~
> Add a new SessionJobState - Failed
> --
>
> Key: FLINK-27370
> URL: https://issues.apache.org/jira/browse/FLINK-27370
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> It will be nice if we can add a new SessionJobState Failed to indicate there
> is an error for the session job.
> {code:java}
> status:
> error: 'The error message'
> jobStatus:
> savepointInfo: {}
> reconciliationStatus:
> reconciliationTimestamp: 0
> state: DEPLOYED {code}
> Reason:
> 1. It will be easier for monitoring
> 2. I have a personal controller to submit session jobs, it will be cleaner to
> check the state by a single field and get the details by the error field.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-27370) Add a new SessionJobState - Failed
[
https://issues.apache.org/jira/browse/FLINK-27370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17528185#comment-17528185
]
Xin Hao commented on FLINK-27370:
-
My request is that the `{{{}status.jobStatus.state`{}}} should be `failed` but
not disappear if the session job submitted occur an error.
> Add a new SessionJobState - Failed
> --
>
> Key: FLINK-27370
> URL: https://issues.apache.org/jira/browse/FLINK-27370
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> It will be nice if we can add a new SessionJobState Failed to indicate there
> is an error for the session job.
> {code:java}
> status:
> error: 'The error message'
> jobStatus:
> savepointInfo: {}
> reconciliationStatus:
> reconciliationTimestamp: 0
> state: DEPLOYED {code}
> Reason:
> 1. It will be easier for monitoring
> 2. I have a personal controller to submit session jobs, it will be cleaner to
> check the state by a single field and get the details by the error field.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-27370) Add a new SessionJobState - Failed
[
https://issues.apache.org/jira/browse/FLINK-27370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527934#comment-17527934
]
Xin Hao commented on FLINK-27370:
-
if the `{{{}success=true/false{}}}` field exists in the `status`, yeah, it is.
But it has been removed?
> Add a new SessionJobState - Failed
> --
>
> Key: FLINK-27370
> URL: https://issues.apache.org/jira/browse/FLINK-27370
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> It will be nice if we can add a new SessionJobState Failed to indicate there
> is an error for the session job.
> {code:java}
> status:
> error: 'The error message'
> jobStatus:
> savepointInfo: {}
> reconciliationStatus:
> reconciliationTimestamp: 0
> state: DEPLOYED {code}
> Reason:
> 1. It will be easier for monitoring
> 2. I have a personal controller to submit session jobs, it will be cleaner to
> check the state by a single field and get the details by the error field.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Created] (FLINK-27370) Add a new SessionJobState - Failed
Xin Hao created FLINK-27370:
---
Summary: Add a new SessionJobState - Failed
Key: FLINK-27370
URL: https://issues.apache.org/jira/browse/FLINK-27370
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
It will be nice if we can add a new SessionJobState Failed to indicate there is
an error for the session job.
{code:java}
status:
error: 'The error message'
jobStatus:
savepointInfo: {}
reconciliationStatus:
reconciliationTimestamp: 0
state: DEPLOYED {code}
Reason:
1. It will be easier for monitoring
2. I have a personal controller to submit session jobs, it will be cleaner to
check the state by a single field and get the details by the error field.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Closed] (FLINK-27277) consist the naming style of SessionJob's state field
[
https://issues.apache.org/jira/browse/FLINK-27277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-27277.
---
Resolution: Not A Problem
> consist the naming style of SessionJob's state field
>
>
> Key: FLINK-27277
> URL: https://issues.apache.org/jira/browse/FLINK-27277
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> {code:java}
> spec:
> clusterId: session-cluster-test
> job:
> state: running
> ...
> status:
> jobStatus:
> state: RUNNING
> ...{code}
> I think it's better to change the spec.job.state with upper case to consist
> of the field in the status spec.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
[jira] [Commented] (FLINK-27277) consist the naming style of SessionJob's state field
[
https://issues.apache.org/jira/browse/FLINK-27277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17524181#comment-17524181
]
Xin Hao commented on FLINK-27277:
-
Thanks for the explanation. Maybe we should consider explicit mark the
difference?
{code:java}
spec:
clusterId: session-cluster-test
job:
state: running
...
status:
state: running
flinkJobStatus:
state: RUNNING {code}
I think the users can figure out the difference if they look at the info
carefully (Maybe thinking for seconds).
But sometimes it's a bit confusing when they take the first glance.
also, the current state should be in the status spec.
> consist the naming style of SessionJob's state field
>
>
> Key: FLINK-27277
> URL: https://issues.apache.org/jira/browse/FLINK-27277
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> {code:java}
> spec:
> clusterId: session-cluster-test
> job:
> state: running
> ...
> status:
> jobStatus:
> state: RUNNING
> ...{code}
> I think it's better to change the spec.job.state with upper case to consist
> of the field in the status spec.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-27277) consist the naming style of SessionJob's state field
Xin Hao created FLINK-27277:
---
Summary: consist the naming style of SessionJob's state field
Key: FLINK-27277
URL: https://issues.apache.org/jira/browse/FLINK-27277
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
{code:java}
spec:
clusterId: session-cluster-test
job:
state: running
...
status:
jobStatus:
state: RUNNING
...{code}
I think it's better to change the spec.job.state with upper case to consist of
the field in the status spec.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (FLINK-27254) Add Volume and VolumeMount supports for FlinkSessionJob
[
https://issues.apache.org/jira/browse/FLINK-27254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-27254.
---
Resolution: Won't Do
> Add Volume and VolumeMount supports for FlinkSessionJob
> ---
>
> Key: FLINK-27254
> URL: https://issues.apache.org/jira/browse/FLINK-27254
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> We should add Volume and VolumeMount supports for the FlinkSessionJob CRD
> The reasons why not do this by PodTemplate are:
> 1. If I only want to mount a volume for the session job, PodTemplate is a bit
> complex
> 2. The volume and mount are dynamic for different session jobs in my scenarios
> The draft CRD will look like
> {code:java}
> apiVersion: flink.apache.org/v1beta1
> kind: FlinkSessionJob
> ...
> spec:
> ...
> job:
> ...
> volumes:
> ...
> volumeMounts:
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-27254) Add Volume and VolumeMount supports for FlinkSessionJob
[
https://issues.apache.org/jira/browse/FLINK-27254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522712#comment-17522712
]
Xin Hao commented on FLINK-27254:
-
I see, thanks
> Add Volume and VolumeMount supports for FlinkSessionJob
> ---
>
> Key: FLINK-27254
> URL: https://issues.apache.org/jira/browse/FLINK-27254
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> We should add Volume and VolumeMount supports for the FlinkSessionJob CRD
> The reasons why not do this by PodTemplate are:
> 1. If I only want to mount a volume for the session job, PodTemplate is a bit
> complex
> 2. The volume and mount are dynamic for different session jobs in my scenarios
> The draft CRD will look like
> {code:java}
> apiVersion: flink.apache.org/v1beta1
> kind: FlinkSessionJob
> ...
> spec:
> ...
> job:
> ...
> volumes:
> ...
> volumeMounts:
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Comment Edited] (FLINK-27254) Add Volume and VolumeMount supports for FlinkSessionJob
[
https://issues.apache.org/jira/browse/FLINK-27254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522697#comment-17522697
]
Xin Hao edited comment on FLINK-27254 at 4/15/22 7:11 AM:
--
*not to a running pod dynamically.*
We only create the volume and mount when creating the pod.
Sorry for not explaining clearly, I mean different volumes for different
SessionJobs so I can't do this in the FlinkDeployment's PodTemplate
was (Author: JIRAUSER286449):
*not to a running pod dynamically.*
We only create the volume and mount when creating the pod.
Sorry for not explaining clearly, I mean different volumes for different
SessionJob so that cant do this in the FlinkDeployment's PodTemplate
> Add Volume and VolumeMount supports for FlinkSessionJob
> ---
>
> Key: FLINK-27254
> URL: https://issues.apache.org/jira/browse/FLINK-27254
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> We should add Volume and VolumeMount supports for the FlinkSessionJob CRD
> The reasons why not do this by PodTemplate are:
> 1. If I only want to mount a volume for the session job, PodTemplate is a bit
> complex
> 2. The volume and mount are dynamic for different session jobs in my scenarios
> The draft CRD will look like
> {code:java}
> apiVersion: flink.apache.org/v1beta1
> kind: FlinkSessionJob
> ...
> spec:
> ...
> job:
> ...
> volumes:
> ...
> volumeMounts:
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-27254) Add Volume and VolumeMount supports for FlinkSessionJob
[
https://issues.apache.org/jira/browse/FLINK-27254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522697#comment-17522697
]
Xin Hao commented on FLINK-27254:
-
*not to a running pod dynamically.*
We only create the volume and mount when creating the pod.
Sorry for not explaining clearly, I mean different volumes for different
SessionJob so that cant do this in the FlinkDeployment's PodTemplate
> Add Volume and VolumeMount supports for FlinkSessionJob
> ---
>
> Key: FLINK-27254
> URL: https://issues.apache.org/jira/browse/FLINK-27254
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> We should add Volume and VolumeMount supports for the FlinkSessionJob CRD
> The reasons why not do this by PodTemplate are:
> 1. If I only want to mount a volume for the session job, PodTemplate is a bit
> complex
> 2. The volume and mount are dynamic for different session jobs in my scenarios
> The draft CRD will look like
> {code:java}
> apiVersion: flink.apache.org/v1beta1
> kind: FlinkSessionJob
> ...
> spec:
> ...
> job:
> ...
> volumes:
> ...
> volumeMounts:
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-27254) Add Volume and VolumeMount supports for FlinkSessionJob
Xin Hao created FLINK-27254:
---
Summary: Add Volume and VolumeMount supports for FlinkSessionJob
Key: FLINK-27254
URL: https://issues.apache.org/jira/browse/FLINK-27254
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
We should add Volume and VolumeMount supports for the FlinkSessionJob CRD
The reasons why not do this by PodTemplate are:
1. If I only want to mount a volume for the session job, PodTemplate is a bit
complex
2. The volume and mount are dynamic for different session jobs in my scenarios
The draft CRD will look like
{code:java}
apiVersion: flink.apache.org/v1beta1
kind: FlinkSessionJob
...
spec:
...
job:
...
volumes:
...
volumeMounts:
...{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (FLINK-27249) Change the state field in the FlinkSessionJob.Spec.Job into desiredState
[
https://issues.apache.org/jira/browse/FLINK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-27249.
---
Resolution: Not A Problem
> Change the state field in the FlinkSessionJob.Spec.Job into desiredState
>
>
> Key: FLINK-27249
> URL: https://issues.apache.org/jira/browse/FLINK-27249
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> {code:java}
> /** Desired state for the job. */
> private JobState state = JobState.RUNNING; {code}
> It will confuse the users when getting info from the K8s. Such as
> {code:java}
> spec:
> clusterId: session-cluster-test
> job:
> jarURI: file:///flink-examples-streaming_2.12-1.14.4-TopSpeedWindowing.jar
> state: running
> ...
> status:
> jobStatus:
> jobId: 42bf3c6f2ac1d909b054b75c20c30afe
> state: RUNNING
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-27249) Change the state field in the FlinkSessionJob.Spec.Job into desiredState
[
https://issues.apache.org/jira/browse/FLINK-27249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522248#comment-17522248
]
Xin Hao commented on FLINK-27249:
-
Agreed with `everything in the Spec is "desired". `
But the `state/status` in the K8s's objects is standing for `current condition`
in most scenarios.
I'm not sure whether there are more desired states but not running, if so, I
think it should be called `mode` or split the state into multi boolean flags
such as `suspend: false`?
Anyway, this is not something big, I'll keep this ticket open for 1 or 2 days
and will close it if no more discussions.
> Change the state field in the FlinkSessionJob.Spec.Job into desiredState
>
>
> Key: FLINK-27249
> URL: https://issues.apache.org/jira/browse/FLINK-27249
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> {code:java}
> /** Desired state for the job. */
> private JobState state = JobState.RUNNING; {code}
> It will confuse the users when getting info from the K8s. Such as
> {code:java}
> spec:
> clusterId: session-cluster-test
> job:
> jarURI: file:///flink-examples-streaming_2.12-1.14.4-TopSpeedWindowing.jar
> state: running
> ...
> status:
> jobStatus:
> jobId: 42bf3c6f2ac1d909b054b75c20c30afe
> state: RUNNING
> ...{code}
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (FLINK-27203) Add supports for uploading jars from object storage (s3, gcs, oss)
[
https://issues.apache.org/jira/browse/FLINK-27203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao closed FLINK-27203.
---
Resolution: Duplicate
> Add supports for uploading jars from object storage (s3, gcs, oss)
> --
>
> Key: FLINK-27203
> URL: https://issues.apache.org/jira/browse/FLINK-27203
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Major
>
> I think it will be efficient if we can read jars from object storage.
>
> * We can detect the object storage provider by path scheme. (Such as:
> `gs://` for GCS)
> * Because the most cloud providers' SDK support native permission
> integration with K8s, for example, the GCP, the user can bind the permission
> by service accounts, and the code running in the K8s cluster will be simple
> `{{{}storage = StorageOptions.getDefaultInstance().service{}}}`
> * In this way, the users only need to bind the service accounts permissions
> and don't need to handle volumes mount and jars initialize (download jars
> from object storage into the volumes by themselves).
> I think we can define the interface first and let the community help
> contribute to the implementation for the different cloud providers.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-27203) Add supports for uploading jars from object storage (s3, gcs, oss)
[
https://issues.apache.org/jira/browse/FLINK-27203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522189#comment-17522189
]
Xin Hao commented on FLINK-27203:
-
Thanks, will watch that ticket~
> Add supports for uploading jars from object storage (s3, gcs, oss)
> --
>
> Key: FLINK-27203
> URL: https://issues.apache.org/jira/browse/FLINK-27203
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Major
>
> I think it will be efficient if we can read jars from object storage.
>
> * We can detect the object storage provider by path scheme. (Such as:
> `gs://` for GCS)
> * Because the most cloud providers' SDK support native permission
> integration with K8s, for example, the GCP, the user can bind the permission
> by service accounts, and the code running in the K8s cluster will be simple
> `{{{}storage = StorageOptions.getDefaultInstance().service{}}}`
> * In this way, the users only need to bind the service accounts permissions
> and don't need to handle volumes mount and jars initialize (download jars
> from object storage into the volumes by themselves).
> I think we can define the interface first and let the community help
> contribute to the implementation for the different cloud providers.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-27249) Change the state field in the FlinkSessionJob.Spec.Job into desiredState
Xin Hao created FLINK-27249:
---
Summary: Change the state field in the FlinkSessionJob.Spec.Job
into desiredState
Key: FLINK-27249
URL: https://issues.apache.org/jira/browse/FLINK-27249
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
{code:java}
/** Desired state for the job. */
private JobState state = JobState.RUNNING; {code}
It will confuse the users when getting info from the K8s. Such as
{code:java}
spec:
clusterId: session-cluster-test
job:
jarURI: file:///flink-examples-streaming_2.12-1.14.4-TopSpeedWindowing.jar
state: running
...
status:
jobStatus:
jobId: 42bf3c6f2ac1d909b054b75c20c30afe
state: RUNNING
...{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-27203) Add supports for uploading jars from object storage (s3, gcs, oss)
Xin Hao created FLINK-27203:
---
Summary: Add supports for uploading jars from object storage (s3,
gcs, oss)
Key: FLINK-27203
URL: https://issues.apache.org/jira/browse/FLINK-27203
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
I think it will be efficient if we can read jars from object storage.
* We can detect the object storage provider by path scheme. (Such as: `gs://`
for GCS)
* Because the most cloud providers' SDK support native permission integration
with K8s, for example, the GCP, the user can bind the permission by service
accounts, and the code running in the K8s cluster will be simple `{{{}storage =
StorageOptions.getDefaultInstance().service{}}}`
* In this way, the users only need to bind the service accounts permissions
and don't need to handle volumes mount and jars initialize (download jars from
object storage into the volumes by themselves).
I think we can define the interface first and let the community help contribute
to the implementation for the different cloud providers.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-27098) The session job controller should create FlinkDeployment informer event source with namespace definition
Xin Hao created FLINK-27098:
---
Summary: The session job controller should create FlinkDeployment
informer event source with namespace definition
Key: FLINK-27098
URL: https://issues.apache.org/jira/browse/FLINK-27098
Project: Flink
Issue Type: Bug
Components: Kubernetes Operator
Reporter: Xin Hao
The below error will occur if we deploy the operator with a namespaced scope
and submit a session job.
{code:java}
[WARN ] Error starting
org.apache.flink.kubernetes.operator.controller.FlinkSessionJobController$1@96a75da
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET
at:
https://10.16.0.1/apis/flink.apache.org/v1alpha1/namespaces/flink-operator/flinkdeployments.
Message: Forbidden!Configured service account doesn't have access. Service
account may have been revoked. flinkdeployments.flink.apache.org is forbidden:
User "system:serviceaccount:flink-operator:flink-operator" cannot list resource
"flinkdeployments" in API group "flink.apache.org" in the namespace
"flink-operator". {code}
This error comes from the creation of the FlinkDeployment informer event source.
I just submitted a PR to fix this,
https://github.com/apache/flink-kubernetes-operator/pull/157
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Resolved] (FLINK-26875) Operator Helm: Extends the rbac options with separate namespaces param
[
https://issues.apache.org/jira/browse/FLINK-26875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xin Hao resolved FLINK-26875.
-
Resolution: Not A Problem
> Operator Helm: Extends the rbac options with separate namespaces param
> --
>
> Key: FLINK-26875
> URL: https://issues.apache.org/jira/browse/FLINK-26875
> Project: Flink
> Issue Type: Improvement
> Components: Kubernetes Operator
>Reporter: Xin Hao
>Priority: Minor
>
> Currently, the Flink Kubernetes Operator Helm Chart uses the
> *watchNamespaces* to determine which namespaces should the roles be created.
> {code:java}
> {{- if .Values.rbac.create }}
> {{- if .Values.watchNamespaces }}
> {{- range .Values.watchNamespaces }}
> ...{code}
> But my use case is that I have another operator for myself to observe the
> Flink Operator's CRs so that I want to create roles in more namespaces.
> So the requirement for me is that add another param called namespaces under
> the *rbac* section and *defaults to watchNamespaces.*
> It will look like this:
> {code:java}
> # values.yaml
> rbac:
> create: true
> # The namespaces should create the roles, defaults to watchNamespaces
> # namespaces: ["flink"]
> # rbac.yaml
> {{- if or .Values.rbac.namespaces .Values.watchNamespaces }}
> {{- range (default .Values.watchNamespaces .Values.rbac.namespaces) }}{code}
> So that the users can create their role bindings.
> The reason that the users shouldn't create roles by themselves is the
> contents of the role should be managed y the Flink Helm chart.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Created] (FLINK-26875) Operator Helm: Extends the rbac options with separate namespaces param
Xin Hao created FLINK-26875:
---
Summary: Operator Helm: Extends the rbac options with separate
namespaces param
Key: FLINK-26875
URL: https://issues.apache.org/jira/browse/FLINK-26875
Project: Flink
Issue Type: Improvement
Components: Kubernetes Operator
Reporter: Xin Hao
Currently, the Flink Kubernetes Operator Helm Chart uses the *watchNamespaces*
to determine which namespaces should the roles be created.
{code:java}
{{- if .Values.rbac.create }}
{{- if .Values.watchNamespaces }}
{{- range .Values.watchNamespaces }}
...{code}
But my use case is that I have another operator for myself to observe the Flink
Operator's CRs so that I want to create roles in more namespaces.
So the requirement for me is that add another param called namespaces under the
*rbac* section and *defaults to watchNamespaces.*
It will look like this:
{code:java}
# values.yaml
rbac:
create: true
# The namespaces should create the roles, defaults to watchNamespaces
# namespaces: ["flink"]
# rbac.yaml
{{- if or .Values.rbac.namespaces .Values.watchNamespaces }}
{{- range (default .Values.watchNamespaces .Values.rbac.namespaces) }}{code}
So that the users can create their role bindings.
The reason that the users shouldn't create roles by themselves is the contents
of the role should be managed y the Flink Helm chart.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (FLINK-26806) Add deployment guide for Argo CD
[ https://issues.apache.org/jira/browse/FLINK-26806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17510985#comment-17510985 ] Xin Hao commented on FLINK-26806: - I just opened a PR for this [https://github.com/apache/flink-kubernetes-operator/pull/98/files] And the below links are the Argo apps working in my cluster https://github.com/haoxins/poc-flink-k8s-native/blob/main/argocd-applications/flink-operator-crds.yaml [https://github.com/haoxins/poc-flink-k8s-native/blob/main/argocd-applications/flink-operator-skip-crds.yaml] > Add deployment guide for Argo CD > > > Key: FLINK-26806 > URL: https://issues.apache.org/jira/browse/FLINK-26806 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > I have deployed the Flink Operator in my cluster with Argo CD. > But there will be an error caused by `CRD too big`. > This is not some bug caused by the operator itself, just the same as this > issue > https://github.com/prometheus-operator/prometheus-operator/issues/4439 > But maybe it's worth adding some guidelines in the document? > We need to tell the users to use Argo CD a bit differently from the other > applications. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-26806) Add deployment guide for Argo CD
[ https://issues.apache.org/jira/browse/FLINK-26806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-26806: External issue URL: https://github.com/apache/flink-kubernetes-operator/pull/98/files > Add deployment guide for Argo CD > > > Key: FLINK-26806 > URL: https://issues.apache.org/jira/browse/FLINK-26806 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > I have deployed the Flink Operator in my cluster with Argo CD. > But there will be an error caused by `CRD too big`. > This is not some bug caused by the operator itself, just the same as this > issue > https://github.com/prometheus-operator/prometheus-operator/issues/4439 > But maybe it's worth adding some guidelines in the document? > We need to tell the users to use Argo CD a bit differently from the other > applications. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-26806) Add deployment guide for Argo CD
[ https://issues.apache.org/jira/browse/FLINK-26806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-26806: Description: I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue https://github.com/prometheus-operator/prometheus-operator/issues/4439 But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. was: I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue [https://github.com/prometheus-operator/prometheus-operator/issues/4439|https://github.com/prometheus-operator/prometheus-operator/issues/4439.] But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. > Add deployment guide for Argo CD > > > Key: FLINK-26806 > URL: https://issues.apache.org/jira/browse/FLINK-26806 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > I have deployed the Flink Operator in my cluster with Argo CD. > But there will be an error caused by `CRD too big`. > This is not some bug caused by the operator itself, just the same as this > issue > https://github.com/prometheus-operator/prometheus-operator/issues/4439 > But maybe it's worth adding some guidelines in the document? > We need to tell the users to use Argo CD a bit differently from the other > applications. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-26806) Add deployment guide for Argo CD
[ https://issues.apache.org/jira/browse/FLINK-26806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-26806: Description: I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue [https://github.com/prometheus-operator/prometheus-operator/issues/4439|https://github.com/prometheus-operator/prometheus-operator/issues/4439.] But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. was: I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue [https://github.com/prometheus-operator/prometheus-operator/issues/4439|https://github.com/prometheus-operator/prometheus-operator/issues/4439.] But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. > Add deployment guide for Argo CD > > > Key: FLINK-26806 > URL: https://issues.apache.org/jira/browse/FLINK-26806 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > I have deployed the Flink Operator in my cluster with Argo CD. > But there will be an error caused by `CRD too big`. > This is not some bug caused by the operator itself, just the same as this > issue > [https://github.com/prometheus-operator/prometheus-operator/issues/4439|https://github.com/prometheus-operator/prometheus-operator/issues/4439.] > But maybe it's worth adding some guidelines in the document? > We need to tell the users to use Argo CD a bit differently from the other > applications. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (FLINK-26806) Add deployment guide for Argo CD
[ https://issues.apache.org/jira/browse/FLINK-26806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin Hao updated FLINK-26806: Description: I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue [https://github.com/prometheus-operator/prometheus-operator/issues/4439|https://github.com/prometheus-operator/prometheus-operator/issues/4439.] But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. was: I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue [https://github.com/prometheus-operator/prometheus-operator/issues/4439.] But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. > Add deployment guide for Argo CD > > > Key: FLINK-26806 > URL: https://issues.apache.org/jira/browse/FLINK-26806 > Project: Flink > Issue Type: Improvement > Components: Kubernetes Operator >Reporter: Xin Hao >Priority: Minor > > I have deployed the Flink Operator in my cluster with Argo CD. > But there will be an error caused by `CRD too big`. > This is not some bug caused by the operator itself, just the same as this > issue > [https://github.com/prometheus-operator/prometheus-operator/issues/4439|https://github.com/prometheus-operator/prometheus-operator/issues/4439.] > But maybe it's worth adding some guidelines in the document? > We need to tell the users to use Argo CD a bit differently from the other > applications. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (FLINK-26806) Add deployment guide for Argo CD
Xin Hao created FLINK-26806: --- Summary: Add deployment guide for Argo CD Key: FLINK-26806 URL: https://issues.apache.org/jira/browse/FLINK-26806 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Xin Hao I have deployed the Flink Operator in my cluster with Argo CD. But there will be an error caused by `CRD too big`. This is not some bug caused by the operator itself, just the same as this issue [https://github.com/prometheus-operator/prometheus-operator/issues/4439.] But maybe it's worth adding some guidelines in the document? We need to tell the users to use Argo CD a bit differently from the other applications. -- This message was sent by Atlassian Jira (v8.20.1#820001)
