Muhammad Hilmi Al Fatih created FLINK-29270: -----------------------------------------------
Summary: Wrong metrics kafka producer (FlinkKafkaProducer, KafkaSink) in EXACTLY_ONCE Key: FLINK-29270 URL: https://issues.apache.org/jira/browse/FLINK-29270 Project: Flink Issue Type: Bug Components: Connectors / Kafka, Runtime / Metrics Affects Versions: 1.14.4 Reporter: Muhammad Hilmi Al Fatih Attachments: image-2022-09-13-11-41-51-759.png, image-2022-09-13-11-42-45-297.png, image-2022-09-13-11-43-38-226.png, image-2022-09-13-11-43-50-441.png I did a very simple kafka-to-kafka pipeline with flink 1.14.4 with checkpoint. I did 4 tests to confirm: for each old API (FlinkKafkaConsumer&FlinkKafkaProducer) and new API (KafkaSource&KafkaSink), I run AT_LEAST_ONCE and EXACLTY_ONCE. However, the metrics shown by producer is not correct when doing EXACTLY_ONCE semantics in both APIs. To understand the situation, I also check the incoming rate metrics for both source and sink topics. So here are the summary: h3. Old API h4. At Least Once !image-2022-09-13-11-41-51-759.png|width=616,height=190! h4. Exactly Once !image-2022-09-13-11-42-45-297.png! h3. New API (KafkaSource&KafkaSink) h4. At Least Once !image-2022-09-13-11-43-38-226.png! h4. Exactly Once !image-2022-09-13-11-43-50-441.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)