Re: Per Partition Watermarking source idleness

2019-08-23 Thread Eduardo Winpenny Tejedor
Hi Prakhar, Everything is probably working as expected, if a partition does not receive any messages then the watermark of the operator does not advance (as it is the minimum across all partitions). You'll need to define a strategy for the watermark to advance even when no messages are received

Per Partition Watermarking source idleness

2019-08-23 Thread Prakhar Mathur
Hi, We are using flink v1.6. We are facing data loss issues while consuming data from older offsets in Kafka with windowing. We are exploring per partition watermarking strategy. But we noticed that when we are trying to consume from multiple topics and if any of the partition is not receiving