AFAIK this metric is directly forwarded from Kafka as-is; so Flink isn't
calculating anything.
I suggest to reach out to the Kafka folks.
On 25/08/2021 17:23, Shilpa Shankar wrote:
Hello ,
We have enabled DataDogHTTPReporter to fetch metrics on flink v1.13.1
running on kubernetes. The metric
flink.operator.KafkaConsumer.records_lag_max is not displaying
accurate values. It also displays 0 most of the time and when it does
fetch a value, it seems to be wrong when I compare them with the Kafka
lag broker metrics.
Could you please let us know how these metrics are calculated? Are
there any configuration changes that need to be made to support the
Kafka Consumer metrics?
# Datadog Integration
metrics.reporter.dghttp.class:
org.apache.flink.metrics.datadog.DatadogHttpReporter
metrics.reporter.dghttp.apikey: xxxx
metrics.reporter.dghttp.maxMetricsPerRequest: 1000
metrics.reporter.dghttp.tags:
flink-cluster:flink-noc-cluster,data-center:lab
metrics.scope.jm <http://metrics.scope.jm>: flink.jobmanager
metrics.scope.jm.job: flink.jobmanager.job
metrics.scope.tm <http://metrics.scope.tm>: flink.taskmanager
metrics.scope.tm.job: flink.taskmanager.job
metrics.scope.task: flink.task
Thanks,
Shilpa
