Helmut Zechmann created FLINK-10300:
---------------------------------------
Summary: Prometheus job level metrics not removed after job
finished
Key: FLINK-10300
URL: https://issues.apache.org/jira/browse/FLINK-10300
Project: Flink
Issue Type: Bug
Components: Metrics
Affects Versions: 1.5.3
Reporter: Helmut Zechmann
Flink provides job level metrics for flink jobs. After a job is finished those
metrics should be removed, else we run into problems when many jobs are
executed on a cluster.
How to reproduce this:
Setup:
* flink 1.5.3 in standalone mode
* 1 JobManager
* 1 TaskManager
* flink-metrics-prometheus-1.5.3.jar in lib dir
Metrics config:
{code:java}
metrics.reporters: prom
metrics.reporter.prom.class:
org.apache.flink.metrics.prometheus.PrometheusReporter
metrics.reporter.prom.port: 7000-7001
{code}
Run the wordcount job. After running the job, job related metrics are still
available:
{code:java}
flink_taskmanager_Status_JVM_GarbageCollector_G1_Old_Generation_Count{tm_id="ee893c28f70d285e701f838706ce8810",host="helmuts_mbp",}
1.0
# HELP flink_taskmanager_job_task_operator_numRecordsOutPerSecond
numRecordsOutPerSecond (scope: taskmanager_job_task_operator)
# TYPE flink_taskmanager_job_task_operator_numRecordsOutPerSecond gauge
flink_taskmanager_job_task_operator_numRecordsOutPerSecond{job_id="2a7c77aacf6b18da389189a3bae6ff48",task_id="529e7a1eaba520b18dc7864f821ada08",task_attempt_id="3bc0d07eb56df676b088a8ec13531c98",host="helmuts_mbp",operator_id="529e7a1eaba520b18dc7864f821ada08",operator_name="DataSource__at_getDefaultTextLineDataSet_WordCountData_java:70___org_apache_flin",task_name="CHAIN_DataSource__at_getDefaultTextLineDataSet_WordCountData_java:70___org_apache_flink_api_java_io_CollectionInputFormat______FlatMap__FlatMap_at_main_WordCount_java:77______Combine__SUM_1___at_main_WordCount_java:80_",task_attempt_num="0",job_name="Flink_Java_Job_at_Fri_Sep_07_13:00:12_CEST_2018",tm_id="ee893c28f70d285e701f838706ce8810",subtask_index="0",}
0.0
...
{code}
With each finished job the prometheus output gets bigger and bigger until the
prometheus output fails to load.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
