It might be possible to apply backpressure to the channels that are
significantly ahead in event time. Tao, it would not be trivial, but if
you'd like to investigate more deeply, take a look at the Flink runtime's
`StatusWatermarkValve` and the associated stream input processors to see
how an
Without throttle, it will eventually ran out of memory.
I think this is a very common use case for Flink users during stream replay
or re-process.
Do we have anything feature planed for it? Would like to contribute on the
initiative.
On Wed, May 2, 2018 at 2:43 AM, Fabian Hueske
Hi Tao,
The watermarks of operators that consume from two (or more) streams are
always synced to the lowest watermark.
This behavior guarantees that data won't be late (unless it was late when
watermarks were assigned). However, the operator will most likely need to
buffer more events from the
Hi All,
I am trying to reply events from 3 different sources and hopefully in
time sequence, say Stream1, Stream2, Stream3. Since their size vary a lot,
the watermarks on one stream is much faster than other streams. Is there
any way to coordinate the watermarks between different input streams.
