Re: Support for Event time clock specific to each stream in parallel streams

2020-08-04 Thread Sush Bankapura
Hi David, Thanks for the quick reply You are indeed right, "Key based watermarking support" in Flink would be most useful I gather this cannot be implemented in Flink as per this mail chain -

Re: Support for Event time clock specific to each stream in parallel streams

2020-07-31 Thread David Anderson
It sounds like you would like to have something like event-time-based windowing, but with independent watermarking for every key. An approach that can work, but it is somewhat cumbersome, is to not use watermarks or windows, but instead put all of the logic in a KeyedProcessFunction (or

Support for Event time clock specific to each stream in parallel streams

2020-07-31 Thread Sush Bankapura
Hi, We have a single Flink job that works on data from multiple data sources. These data sources are not aligned in time and also have intermittent connectivity lasting for days, due to which data will arrive late We attempted to use the event time and watermarks with parallel streams using