Are configs stored as part of savepoints

2021-04-19 Thread gaurav kulkarni
Hi,  I was wondering if configs applied while creating a flink application are also stored as part of savepoint? If yes, an app is restored from a savepoint, does it start with the same configs? Thanks 

CRD compatible with native and standalone mode

2021-04-19 Thread gaurav kulkarni
Hi,  I plan to create a flink K8s operator which supports standalone mode, and and switch to native mode sometime later. I was wondering what are some of the approaches to ensure that CRD is compatible with both native and standalone mode?  Thanks 

Re: CRD compatible with native and standalone mode

2021-04-20 Thread gaurav kulkarni
ke this[1]. Since the CR is defined in yaml[2], native and standalone could have some dedicated fields.And you could easily parse them in your K8s operator. [1].  https://github.com/wangyang0918/flink-native-k8s-operator/blob/master/deploy/crd.yaml[2].   https://github.com/wangyang0918/flink-native-k8s

Official flink java client

2021-04-22 Thread gaurav kulkarni
Hi,  Is there any official flink client in java that's available? I came across RestClusterClient, but I am not sure if its official. I can create my own client, but just wanted to check if there is anything official available already that I can leverage.  Thanks,Gaurav | | | | | | | |

Re: Official flink java client

2021-04-23 Thread gaurav kulkarni
Thanks for the response, folks! I plan to use the client mostly for monitoring status of jobs, probably to trigger savepoints too. I may extend it in future to submit jobs. Given RestClusterClient is not officially supported, I will probably build something myself. Agree with Flavio, it would b

Identify metrics belonging to the "same" task manager in kubernetes

2021-08-25 Thread gaurav kulkarni
Hi,  We have multiple flink clusters running in kubernetes. We plan to enable prometheus on these clusters. Looks like flink metrics emitted are of the format: "flink_taskmanager_Status_JVM_GarbageCollector_G1_Young_Generation_Time{host="10_244_2_6",tm_id="10_244_2_6:6122_2e3d7a",} 65.0" 1.  Si

Re: Identify metrics belonging to the "same" task manager in kubernetes

2021-08-26 Thread gaurav kulkarni
, without being tied to a specific process? 2) It is not possible for the PrometheusReporter. On 26/08/2021 04:22, gaurav kulkarni wrote: Hi,  We have multiple flink clusters running in kubernetes. We plan to enable prometheus on these clusters. Looks like flink metrics e

Re: Identify metrics belonging to the "same" task manager in kubernetes

2021-08-26 Thread gaurav kulkarni
On 26/08/2021 16:46, gaurav kulkarni wrote: Thanks for the response!  1) That's correct. I was wondering if there is any logical ID for task manager that stays the same even if its moved to a different pod and is a part of metrics emitted. The scenario is if a TM#1 was running on pod#

Re: Identify metrics belonging to the "same" task manager in kubernetes

2021-08-26 Thread gaurav kulkarni
Hi,  I have another question: What mechanisms are usually used to correlate prometheus flink metrics for kubernetes?  Thanks,Gaurav  On Thursday, August 26, 2021, 10:06:30 AM PDT, gaurav kulkarni wrote: Thanks for the response! For #2, custom labels should work too for our case