[ https://issues.apache.org/jira/browse/FLINK-27944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-27944: ----------------------------------- Labels: pull-request-available stale-assigned (was: pull-request-available) I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help the community manage its development. I see this issue is assigned but has not received an update in 30 days, so it has been labeled "stale-assigned". If you are still working on the issue, please remove the label and add a comment updating the community on your progress. If this issue is waiting on feedback, please consider this a reminder to the committer/reviewer. Flink is a very active project, and so we appreciate your patience. If you are no longer working on the issue, please unassign yourself so someone else may work on it. > IO metrics collision happens if a task has union inputs > ------------------------------------------------------- > > Key: FLINK-27944 > URL: https://issues.apache.org/jira/browse/FLINK-27944 > Project: Flink > Issue Type: Bug > Components: Runtime / Metrics > Affects Versions: 1.15.0 > Reporter: Zhu Zhu > Assignee: xiaogang zhou > Priority: Major > Labels: pull-request-available, stale-assigned > > When a task has union inputs, some IO metrics(numBytesIn* and numBuffersIn*) > of the different inputs may collide and failed to be registered. > > The problem can be reproduced with a simple job like: > {code:java} > DataStream<String> source1 = env.fromElements("abc"); > DataStream<String> source2 = env.fromElements("123"); > source1.union(source2).print();{code} > > Logs of collisions: > {code:java} > 2022-06-08 00:59:01,629 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInLocal'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0, > Shuffle, Netty, Input] > 2022-06-08 00:59:01,629 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInLocalPerSecond'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0, > Shuffle, Netty, Input] > 2022-06-08 00:59:01,629 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInLocal'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0] > 2022-06-08 00:59:01,629 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInLocalPerSecond'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInRemote'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0, > Shuffle, Netty, Input] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInRemotePerSecond'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0, > Shuffle, Netty, Input] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInRemote'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBytesInRemotePerSecond'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInLocal'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0, > Shuffle, Netty, Input] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInLocalPerSecond'. Metric will not be reported.[, > taskmanager, fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to > Std. Out, 0, Shuffle, Netty, Input] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInLocal'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInLocalPerSecond'. Metric will not be reported.[, > taskmanager, fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to > Std. Out, 0] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInRemote'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0, > Shuffle, Netty, Input] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInRemotePerSecond'. Metric will not be reported.[, > taskmanager, fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to > Std. Out, 0, Shuffle, Netty, Input] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInRemote'. Metric will not be reported.[, taskmanager, > fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to Std. Out, 0] > 2022-06-08 00:59:01,630 WARN org.apache.flink.metrics.MetricGroup > [] - Name collision: Group already contains a Metric with the > name 'numBuffersInRemotePerSecond'. Metric will not be reported.[, > taskmanager, fa9f270e-e904-4f69-8227-8d6e26e1be62, WordCount, Sink: Print to > Std. Out, 0] > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)