Hi Timo,
Yes I have gone through the link. But for the other metrics documentation has
description.
For example,
numBytesOut - The total number of bytes this task has emitted.
lastCheckpointSize - The total size of the last checkpoint (in bytes).
For the latency metrics I don't see such description due to which it is
difficult to understand what is the count listed for each operator, how it is
incrementing and values.
It will be helpful if some more information is provided regarding these metrics.
-Original Message-
From: Timo Walther
Sent: Friday, March 5, 2021 8:52 PM
To: user@flink.apache.org
Subject: Re: Need information on latency metrics
Hi Suchithra,
did you see this section in the docs?
https://ci.apache.org/projects/flink/flink-docs-stable/ops/metrics.html#latency-tracking
Regards,
Timo
On 05.03.21 15:31, V N, Suchithra (Nokia - IN/Bangalore) wrote:
> Hi,
>
> I am using flink 1.12.1 version and trying to explore latency metrics
> with Prometheus. I have enabled latency metrics by adding
> *"metrics.latency.interval: 1" *in flink-conf.yaml.
>
> I have submitted a flink streaming job which has
> Source->flatmap->process->sink which is chained into single task. And
> Source->flatmap->process->I
> can see below latency metrics in Prometheus.
>
> flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_i
> ndex_latency_count
>
> flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_i
> ndex_latency
>
> Prometheus output :
>
> *flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency_count*
>
> **
>
> /flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency_count{app="met-flink-taskmanager",
> host="", instance=" ", job="kubernetes-pods-insecure",
> job_id="3ad0b4c814836aea92c48f6baf44b8bb", job_name=" ",
> kubernetes_pod_name="met-flink-taskmanager-5b58cdf557-l24tc",
> namespace="", operator_id="3d05135cf7d8f1375d8f655ba9d20255",
> operator_subtask_index="0", pod_template_hash="5b58cdf557",
> source_id="cbc357ccb763df2852fee8c4fc7d55f2", tm_id=" "}/
>
> */27804583/*
>
> /flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency_count{app="met-flink-taskmanager",
> host="", instance=" ", job="kubernetes-pods-insecure",
> job_id="3ad0b4c814836aea92c48f6baf44b8bb", job_name=" ",
> kubernetes_pod_name="met-flink-taskmanager-5b58cdf557-l24tc",
> namespace=" ", operator_id="570f707193e0fe32f4d86d067aba243b",
> operator_subtask_index="0", pod_template_hash="5b58cdf557",
> source_id="cbc357ccb763df2852fee8c4fc7d55f2", tm_id=" "}/
>
> */27804583/*
>
> /flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency_count{app="met-flink-taskmanager",
> host="", instance=" ", job="kubernetes-pods-insecure",
> job_id="3ad0b4c814836aea92c48f6baf44b8bb", job_name=" ",
> kubernetes_pod_name="met-flink-taskmanager-5b58cdf557-l24tc",
> namespace=" ", operator_id="ba40499bacce995f15693b1735928377",
> operator_subtask_index="0", pod_template_hash="5b58cdf557",
> source_id="cbc357ccb763df2852fee8c4fc7d55f2", tm_id=" "}/
>
> */27804583/*
>
> *//*
>
> *flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency*
>
> /flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency{app="met-flink-taskmanager",
> host="", instance=" ", job="kubernetes-pods-insecure",
> job_id="3ad0b4c814836aea92c48f6baf44b8bb", job_name=" ",
> kubernetes_pod_name="met-flink-taskmanager-5b58cdf557-l24tc",
> namespace=" ", operator_id="3d05135cf7d8f1375d8f655ba9d20255",
> operator_subtask_index="0", pod_template_hash="5b58cdf557",
> quantile="0.95", source_id="cbc357ccb763df2852fee8c4fc7d55f2", tm_id="
> "}/
>
> */0/*
>
> /flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency{app="met-flink-taskmanager",
> host="", instance=" ", job="kubernetes-pods-insecure",
> job_id="3ad0b4c814836aea92c48f6baf44b8bb", job_name=" ",
> kubernetes_pod_name="met-flink-taskmanager-5b58cdf557-l24tc",
> namespace=" ", operator_id="3d05135cf7d8f1375d8f655ba9d20255",
> operator_subtask_index="0", pod_template_hash="5b58cdf557",
> quantile="0.98", source_id="cbc357ccb763df2852fee8c4fc7d55f2", tm_id="
> "}/
>
> */0.4217/*
>
> /flink_taskmanager_job_latency_source_id_operator_id_operator_subtask_
> index_latency{app="met-flink-taskmanager",
> host="", instance=" ", job="kubernetes-pods-insecure",
> job_id="3ad0b4c814836aea92c48f6baf44b8bb", job_name=" ",
> kubernetes_pod_name="met-flink-taskmanager-5b58cdf557-l24tc",
> namespace=" ", operator_id="3d05135cf7d8f1375d8f655ba9d20255",
> operator_subtask_index="0", pod_template_hash="5b58cdf557",
> quantile="0.99", source_id="cbc357ccb763df2852fee8c4fc7d55f2", tm_id="
> "}/
>
> */1/*
>
> *//*
>
> Could someone please explain what is the values reported here for
> these metrics.
>
> Thanks
>
> Suchithra
>
