Re: KafkaSource metrics

2021-05-26 Thread Alexey Trenikhun
Found https://issues.apache.org/jira/browse/FLINK-22766 From: Alexey Trenikhun Sent: Tuesday, May 25, 2021 3:25 PM To: Ardhani Narasimha ; 陳樺威 ; Flink User Mail List Subject: Re: KafkaSource metrics Looks like when KafkaSource is used instead

Re: KafkaSource metrics

2021-05-25 Thread Alexey Trenikhun
Looks like when KafkaSource is used instead of FlinkKafkaConsumer, metrics listed below are not available. Bug? Work in progress? Thanks, Alexey From: Ardhani Narasimha Sent: Monday, May 24, 2021 9:08 AM To: 陳樺威 Cc: user Subject: Re: KafkaSource metrics Use

Re: KafkaSource metrics

2021-05-25 Thread Qingsheng Ren
Hi Oscar, Thanks for raising this problem! Currently metrics of KafkaConsumer are not registered in Flink as in FlinkKafkaConsumer. A ticket has been created on JIRA, and hopefully we can fix it in next release. https://issues.apache.org/jira/browse/FLINK-22766 -- Best Regards, Qingsheng Ren

Re: KafkaSource metrics

2021-05-25 Thread 陳樺威
Hi Ardhani, Thanks for your kindly reply. Our team use your provided metrics before, but the metrics disappear after migrate to new KafkaSource. We initialize KafkaSource in following code. ``` val consumer: KafkaSource[T] = KafkaSource.builder() .setProperties(properties)

Re: KafkaSource metrics

2021-05-24 Thread Ardhani Narasimha
Use below respectively flink_taskmanager_job_task_operator_KafkaConsumer_bytes_consumed_rate - Consumer rate flink_taskmanager_job_task_operator_KafkaConsumer_records_lag_max - Consumer lag flink_taskmanager_job_task_operator_KafkaConsumer_commit_latency_max - commit latency unsure if reactive