> I think this wouldn't be very robust to different situations where processing time and event time may not be that close to each other.
if you do something like `min(endOfWindow, max(eventInputTimestamp, computedFiringTimestamp))` the worst case is that you set a watermark hold for somewhere in the future, right? For example, if the watermark is lagging 3 hours, processing time = 4pm, event input = 1pm, window end = 5pm, the watermark hold/output time is set to 4pm + T. This would make the timestamps "newer" than the input, but shouldn't ever create late data, correct? Also, imo, the timestamps really already cross domains now, because the watermark (event time) is held until the (processing time) timer fires. The concrete issue that brought this up was a pipeline with some state, and the state was "cleaned up" periodically with a processing time timer that fired every ~hour. The author of the pipeline was confused why the watermark wasn't moving (and thus GBKs firing, etc). The root cause was the watermark being held by the timer. > It would just save you .withOutputTimestamp(elementTimestamp) on your calls to setting the event time timer, right? Correct, the main thing I'm trying to solve is having to recalculate an output timestamp using the same logic that the timer itself is using to set its firing timestamp. On Tue, Dec 14, 2021 at 4:10 PM Kenneth Knowles <[email protected]> wrote: > > > On Tue, Dec 7, 2021 at 7:27 AM Steve Niemitz <[email protected]> wrote: > >> If I have a processing time timer, is there any way to automatically set >> the output timestamp to the timer firing timestamp (similar to how >> event-time timers work). >> >> A common use case would be to do something like: >> timer.offset(X).align(Y).setRelative() >> > > > but have the output timestamp be the firing timestamp. In order to do >> this now you need to re-calculate the output timestamp (using the same >> logic as the timer does internally) and manually use withOutputTimestamp. > > > I think this wouldn't be very robust to different situations where > processing time and event time may not be that close to each other. In > general I'm skeptical of reusing timestamps across time domains, for just > this sort of reason. I wouldn't recommend doing this manually either. > > >> I'm not sure what the API would look like here, but it would also be nice >> to allow event-time timers to do the same in reverse (use the element >> input timestamp rather than the firing timestamp). Maybe something like >> `withDefaultOutputTimestampFrom(...)` and an enum of FIRING_TIMESTAMP, >> ELEMENT_TIMESTAMP? >> > > It would just save you .withOutputTimestamp(elementTimestamp) on your > calls to setting the event time timer, right? It doesn't work in general > because a timer can be set from other OnTimer methods, where there is no > "element" per se, but just the output timestamp of the fired timer. > > Kenn >
