Hi Yanfei, Thanks for the feedback!
> 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2 > processes a watermark, the watermark will be sent to downstream, if > the `InternalTimerServiceImpl#advanceWatermark` is interrupted, when > is the watermark sent downstream? The watermark would be outputted by an operator only once all relevant timers are fired. In other words, if firing of timers is interrupted a continuation mail to continue firing those interrupted timers is created. Watermark will be emitted downstream at the end of that continuation mail. > 2. IIUC, processing-timer's firing is also encapsulated into mail and > executed in mailbox. Is processing-timer allowed to be interrupted? Yes, both firing processing and even time timers share the same code and both will support interruptions in the same way. Actually I've renamed the FLIP from > Interruptible watermarks processing to: > Interruptible timers firing to make this more clear. Best, Piotrek wt., 30 kwi 2024 o 06:08 Yanfei Lei <fredia...@gmail.com> napisał(a): > Hi Piotrek, > > Thanks for this proposal. It looks like it will shorten the checkpoint > duration, especially in the case of back pressure. +1 for it! I'd > like to ask some questions to understand your thoughts more precisely. > > 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2 > processes a watermark, the watermark will be sent to downstream, if > the `InternalTimerServiceImpl#advanceWatermark` is interrupted, when > is the watermark sent downstream? > 2. IIUC, processing-timer's firing is also encapsulated into mail and > executed in mailbox. Is processing-timer allowed to be interrupted? > > Best regards, > Yanfei > > Piotr Nowojski <pnowoj...@apache.org> 于2024年4月29日周一 21:57写道： > > > > > Hi all, > > > > I would like to start a discussion on FLIP-443: Interruptible watermark > > processing. > > > > https://cwiki.apache.org/confluence/x/qgn9EQ > > > > This proposal tries to make Flink's subtask thread more responsive when > > processing watermarks/firing timers, and make those operations > > interruptible/break them apart into smaller steps. At the same time, the > > proposed solution could be potentially adopted in other places in the > code > > base as well, to solve similar problems with other flatMap-like operators > > (non windowed joins, aggregations, CepOperator, ...). > > > > I'm looking forward to your thoughts. > > > > Best, > > Piotrek >
