Removing labels from Flink metrics

2023-01-08 Thread Surendra Lalwani via user
Hi Team, Is it possible to remove a few labels from Flink operator scope metrics as we are noticing that sometimes those labels are too large and hence causing unnecessary load at our monitoring platform. One such label is operator_name. Thanks and Regards , Surendra Lalwani -- IMPORTANT

Metrics exposed by flink containing long label values

2023-01-06 Thread Surendra Lalwani via user
Hi Team, We are exposing metrics by Flink to prometheus but we are seeing it contains various labels even including operator details. and task_name, do we have any way from which we can remove these extra / unnecessary labels? Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This

KarkaConsumer_records_lag_max metrics showing weird numbers

2022-12-13 Thread Surendra Lalwani via user
Hi Team, I am on Flink version 1.13.6. I am reading couple of streams from Kafka and applying interval join with interval of 2 hours. However when I am checking KafkaConsumer_records_lag_max it is coming in some thousands but when I check Flink UI there is no backpressure and also the metrics

Performing left join between two streams

2022-10-28 Thread Surendra Lalwani via user
Hi Team, Is it possible in Flink to perform Left Outer join between two streams as it is possible in Spark. I checked for Internal join it only supports inner join as of now. Thanks and Regards , Surendra Lalwani -- IMPORTANT NOTICE: This e-mail, including any attachments, may contain

Re: How to perform something on Checkpointing and Savepointing

2022-09-12 Thread Surendra Lalwani via user
ner. > > Best, > Ben > > > On Mon, Sep 12, 2022 at 4:01 PM Surendra Lalwani via user < > user@flink.apache.org> wrote: > >> Hi Team, >> >> I want to perform some customized operations on checkpointing and >> savepointing. For example, I want

How to perform something on Checkpointing and Savepointing

2022-09-12 Thread Surendra Lalwani via user
Hi Team, I want to perform some customized operations on checkpointing and savepointing. For example, I want to create an additional file every time checkpointing and savepointing gets triggered. Do we have anything as such? Any listeners or anything else? Thanks and Regards , Surendra Lalwani