Thanks for the response!
For #2, custom labels should work too for our case.
Thanks,Gaurav
On Thursday, August 26, 2021, 08:28:27 AM PDT, Chesnay Schepler
<ches...@apache.org> wrote:
1) As is there is no way to accomplish this.
2) Yes (datadog, graphite) but if you are happy with Prometheus I instead
recommend to fork the reporter and adjust it accordingly. Would only custom
prefixes/suffixes work for your use-case, or would a custom label also work?
3) Sure, there's plenty of other
metrics:https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#io
On 26/08/2021 16:46, gaurav kulkarni wrote:
Thanks for the response!
1) That's correct. I was wondering if there is any logical ID for task
manager that stays the same even if its moved to a different pod and is a part
of metrics emitted. The scenario is if a TM#1 was running on pod#1 and then is
moved to pod#2 and if we can correlate the metrics. Please let me know if there
is a way correlate metrics.
2) Are there any other reporters where it is possible? 3) I have one more
question. Looks like most of the metrics emitted are around resource usage
(memory/cpu). Does flink emit any metrics w.r.t job (for example, records
processed, processing rate, overall latency etc)?
Thanks, Gaurav On Thursday, August 26, 2021, 04:02:10 AM PDT, Chesnay
Schepler <ches...@apache.org> wrote:
1) Can you clarify what you mean with "same"? Are you referring to
something like an index, i.e., "This is TaskManager #2", without being tied to
a specific process?
2) It is not possible for the PrometheusReporter.
On 26/08/2021 04:22, gaurav kulkarni wrote:
Hi,
We have multiple flink clusters running in kubernetes. We plan to enable
prometheus on these clusters. Looks like flink metrics emitted are of the
format:
"flink_taskmanager_Status_JVM_GarbageCollector_G1_Young_Generation_Time{host="10_244_2_6",tm_id="10_244_2_6:6122_2e3d7a",}
65.0"
1. Since task managers can get moved across multiple pods, is there is a way
to uniquely identify metrics coming from the "same" task manager but moved to
different pods?
2. Is it possible to add formatting to the names of the metrics (for example,
add any suffix/prefix to the names).
Appreciate your help.
Thanks, Gaurav
