Re: Latency monitoring in Flink 1.14.0

2021-12-13 Thread Timo Walther
we confirm that it has been implemented? Regards, Morgan. *From:* Timo Walther *Sent:* 13 December 2021 09:53:08 *To:* user@flink.apache.org *Subject:* Re: Latency monitoring in Flink 1.14.0 Hi Morgan, did you see th

Re: Latency monitoring in Flink 1.14.0

2021-12-13 Thread Timo Walther
*To:* user@flink.apache.org *Subject:* Re: Latency monitoring in Flink 1.14.0 Hi Morgan, did you see this: https://stackguides.com/questions/68917956/read-flink-latency-tracking-metric-in-datadog <https://stackguides.com/questions/68917956/read-flink-latency-tracking-metric-in-da

Re: Latency monitoring in Flink 1.14.0

2021-12-13 Thread Geldenhuys, Morgan Karl
/KafkaSink these metrics are just not being generated. Can we confirm that it has been implemented? Regards, Morgan. From: Timo Walther Sent: 13 December 2021 09:53:08 To: user@flink.apache.org Subject: Re: Latency monitoring in Flink 1.14.0 Hi Morgan, did you

Re: Latency monitoring in Flink 1.14.0

2021-12-13 Thread Timo Walther
wrote: Greetings all, I am attempting to setup latency monitoring for a flink 1.14.0 job. According to the documentation <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/metrics/>, I have done the following: In my kubernetes setup I have added the following

Latency monitoring in Flink 1.14.0

2021-12-10 Thread Geldenhuys, Morgan Karl
Greetings all, I am attempting to setup latency monitoring for a flink 1.14.0 job. According to the documentation<https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/metrics/>, I have done the following: In my kubernetes setup I have added the following to the kube

Re: latency monitoring

2020-12-10 Thread Fanbin Bu
timestamp and report the latency. > > > > Julian > > > > *From: *Fanbin Bu > *Date: *Thursday, December 10, 2020 at 3:41 PM > *To: *user > *Subject: *latency monitoring > > > > Hi, > > > > I would like to monitor the pipeline latency meas

Re: latency monitoring

2020-12-10 Thread Jaffe, Julian
PM To: user Subject: latency monitoring Hi, I would like to monitor the pipeline latency measured by timestamp when writing the output to sink - timestamp when ingested from the source. Now I'm able to get the timestamp writing to sink since the sink is implementing a RichSinkFunctio

latency monitoring

2020-12-10 Thread Fanbin Bu
Hi, I would like to monitor the pipeline latency measured by timestamp when writing the output to sink - timestamp when ingested from the source. Now I'm able to get the timestamp writing to sink since the sink is implementing a RichSinkFunction and therefore I can report gauge there [1]. But I h

Re: Latency Monitoring in Flink application

2019-06-13 Thread Timothy Victor
you will get a good picture of how the even-time > lag changes over time. > > Hope this helps. > > Cheers, > > Konstantin > > On Thu, Jun 13, 2019 at 12:20 PM Halfon, Roey wrote: > >> Hi All, >> I'm looking for help regarding latency monitoring. >>

Re: Latency Monitoring in Flink application

2019-06-13 Thread Konstantin Knauf
PM Halfon, Roey wrote: > Hi All, > I'm looking for help regarding latency monitoring. > > Let's say I have a simple streaming data flow with the following operators: > FlinkKafkaConsumer -> Map -> print. > > In case I want to measure a latency of records proc

Latency Monitoring in Flink application

2019-06-13 Thread Halfon, Roey
Hi All, I'm looking for help regarding latency monitoring. Let's say I have a simple streaming data flow with the following operators: FlinkKafkaConsumer -> Map -> print. In case I want to measure a latency of records processing in my dataflow, what would be the best opportunit