Hi all, I confused with watermark for each Kafka partitions. As I know watermark created by data stream level. But why also say created watermark for each Kafka topic partitions ? As I tested, watermarks also created by global, even I run my job with parallels. And assign watermarks on Kafka consumer . Thanks .
Below text copied from flink web. you can use Flinkās Kafka-partition-aware watermark generation. Using that feature, watermarks are generated inside the Kafka consumer, per Kafka partition, and the per-partition watermarks are merged in the same way as watermarks are merged on stream shuffles. For example, if event timestamps are strictly ascending per Kafka partition, generating per-partition watermarks with the ascending timestamps watermark generator <https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/event_timestamp_extractors.html#assigners-with-ascending-timestamps> will result in perfect overall watermarks. The illustrations below show how to use the per-Kafka-partition watermark generation, and how watermarks propagate through the streaming dataflow in that case. Thanks Alex Fu