Hm, that would probably work, thanks!
But, should the timers behave like that? I'm trying to fix tris by
introducing a sequence of watermarks
inputs watermark -> timer watermark -> output watermark
as suggested in the JIRA, and it actually seems to be working as
expected. It even cleans some code paths, but I'm debugging some strange
behavior this exposed -
`WatermarkHold.watermarkHoldTagForTimestampCombiner` seems to have
stopped clearing itself after this change and some Pipelines therefore
stopped working. I'm little lost why this happened. I can push code I
have if anyone interested.
Jan
On 6/10/19 5:32 PM, Lukasz Cwik wrote:
We hit an instance of this problem before and solved it rescheduling
the GC timer again if there was a conflicting timer that was also
meant to fire.
On Mon, Jun 10, 2019 at 8:17 AM Jan Lukavský <[email protected]
<mailto:[email protected]>> wrote:
For a single key. I'm getting into collision of timerId
`__StatefulParDoGcTimerId` (StatefulDoFnRunner) and my timerId for
flushing sorted elements in implementation of
@RequiresTimeSortedInput. The timers are being swapped at the end
of input (but it can happen anywhere near end of window), which
results in state being cleared before it gets flushed, which means
data loss.
Jan
On 6/10/19 5:08 PM, Reuven Lax wrote:
Do you mean for a single key or across keys?
On Mon, Jun 10, 2019, 5:11 AM Jan Lukavský <[email protected]
<mailto:[email protected]>> wrote:
Hi,
I have come across issue [1], where I'm not sure how to solve
this in
most elegant way.
Any suggestions?
Thanks,
Jan
[1] https://issues.apache.org/jira/browse/BEAM-7520
