Kanthi Vaidya created FLINK-33224: ------------------------------------- Summary: Flink Batch Job does not complete when using Kafka topics with empty partitions Key: FLINK-33224 URL: https://issues.apache.org/jira/browse/FLINK-33224 Project: Flink Issue Type: Bug Components: Connectors / Kafka Affects Versions: 1.14.6 Reporter: Kanthi Vaidya
Kafka Source Connector no longer throws an exception when running a flink job in batch mode when using Kafka topics with empty partitions. However, flink does not emit the final watermark of Long.MAX_VALUE even if all the messages have been read. Note that we are using bounded sources, since that is a requirement for batch mode. It appears that the Kafka consumer continues to run even though the source is bounded. Same is the case with an empty topic. We do have optional sources, where data need not be present all the time. However if we mark the source as bounded, the Kafka connector should recognize empty partitions and mark it as completed. -- This message was sent by Atlassian Jira (v8.20.10#820010)