Re: Flink Key based watermarks with event time

2020-05-18 Thread Arvid Heise
Watermarks are maintained for each source separately, but need to be combined when you combine the different streams. If not, you would not be able to perform windows and any related join or aggregation. Having different timestamps in the sources is actually the normal case and the main motivation

Re: Flink Key based watermarks with event time

2020-05-15 Thread Congxian Qiu
Hi Maybe you can try KeyedProcessFunction[1] for this, but you need to handle the allow-latency logic[2] in your own business logic(event-time records maybe out-of-order) [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/operators/process_function.html#the-keyedprocessfu

Flink Key based watermarks with event time

2020-05-13 Thread Gnanasoundari Soundarajan
Hi all, We have a requirement where we need to maintain key based watermarks with event time. Each sensor will communicate with different timestamp where we need to maintain watermark separately for each sensor. Is this requirement can be achieved with Flink? Thanks. Regards, Gnana