On Wed, Jan 19, 2022 at 1:11 AM Jan Lukavský wrote:
> > One note - some people definitely use timer.withOutputTimestamp as a
> watermark hold.
>
> Definitely.
>
> > In fact, I do not view a "watermark hold" as a fundamental concept. The
> act of "set a timer with the intent that I am allowed to p
> One note - some people definitely use timer.withOutputTimestamp as a
watermark hold.
Definitely.
> In fact, I do not view a "watermark hold" as a fundamental concept.
The act of "set a timer with the intent that I am allowed to produce
output with timestamp X" is the fundamental concept, an
Yea, it makes sense. This is an issue for the global window where there
isn't automatic cleanup of state. I've had a few user cases where they
would like a good way of doing state cleanup in the global window too -
something where whenever state gets buffer there is always a finite timer
that will
This is an interesting case, and a legitimate counterexample to consider.
I'd call it a workaround :-). The semantic thing they would want/need is
"output timestamp" associated with buffered data (also implemented with
watermark hold). I do know systems that designed their state with this
built in.
On Tue, Dec 14, 2021 at 2:38 PM Steve Niemitz wrote:
> > I think this wouldn't be very robust to different situations where
> processing time and event time may not be that close to each other.
>
> if you do something like `min(endOfWindow, max(eventInputTimestamp,
> computedFiringTimestamp))` th
> I think this wouldn't be very robust to different situations where
processing time and event time may not be that close to each other.
if you do something like `min(endOfWindow, max(eventInputTimestamp,
computedFiringTimestamp))` the worst case is that you set a watermark hold
for somewhere in t
On Tue, Dec 7, 2021 at 7:27 AM Steve Niemitz wrote:
> If I have a processing time timer, is there any way to automatically set
> the output timestamp to the timer firing timestamp (similar to how
> event-time timers work).
>
> A common use case would be to do something like:
> timer.offset(X).ali
If I have a processing time timer, is there any way to automatically set
the output timestamp to the timer firing timestamp (similar to how
event-time timers work).
A common use case would be to do something like:
timer.offset(X).align(Y).setRelative()
but have the output timestamp be the firing
