Hi all,
I confused with watermark for each Kafka partitions. As I know watermark
created by data stream level. But why also say created watermark for each
Kafka topic partitions ? As I tested, watermarks also created by global, even I
run my job with parallels. And assign watermarks on Kafka consumer . Thanks .
Below text copied from flink web.
you can use Flink’s Kafka-partition-aware watermark generation. Using that
feature, watermarks are generated inside the Kafka consumer, per Kafka
partition, and the per-partition watermarks are merged in the same way as
watermarks are merged on stream shuffles.
For example, if event timestamps are strictly ascending per Kafka partition,
generating per-partition watermarks with the ascending timestamps watermark
generator
<https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/event_timestamp_extractors.html#assigners-with-ascending-timestamps>
will result in perfect overall watermarks.
The illustrations below show how to use the per-Kafka-partition watermark
generation, and how watermarks propagate through the streaming dataflow in that
case.
Thanks
Alex Fu
