Kanthi Vaidya created FLINK-33224:
-------------------------------------
Summary: Flink Batch Job does not complete when using Kafka
topics with empty partitions
Key: FLINK-33224
URL: https://issues.apache.org/jira/browse/FLINK-33224
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Affects Versions: 1.14.6
Reporter: Kanthi Vaidya
Kafka Source Connector no longer throws an exception when running a flink job
in batch mode when using Kafka topics with empty partitions. However, flink
does not emit the final watermark of Long.MAX_VALUE even if all the messages
have been read. Note that we are using bounded sources, since that is a
requirement for batch mode. It appears that the Kafka consumer continues to run
even though the source is bounded.
Same is the case with an empty topic. We do have optional sources, where data
need not be present all the time. However if we mark the source as bounded, the
Kafka connector should recognize empty partitions and mark it as completed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
