reuvenlax commented on PR #22347: URL: https://github.com/apache/beam/pull/22347#issuecomment-1197311112
Ah I see what the confusion is. This timer is set using the withNoOutputTimestamp, largely because otherwise the watermark hold causes problems. That was the source of the original bug, because we shouldn't be outputting from a timer set like that. A timer set using withNoOutputTimestamp has the default output timestamp set to an invalid value as you noticed. This fix is a hack - explicitly setting the production timestamp to a valid value (end of window) and then overriding the allowed timestamp skew to a value >= 1ms allows the production to go through. However it is sketchy - it relies on knowing how withNoOutputTimestamp is implemented. The "correct" fix would require more significant refactoring. Given that this is a regression that makes the sink unusable for many users, I wanted to get a fix into the current beam release first and then replace this with a more-principled fix (which would be to do the cleanup inline in onTimer instead of delegating to another ParDo). On Wed, Jul 27, 2022 at 12:54 PM Kenn Knowles ***@***.***> wrote: > I think that creation of the window must not be validated, or some such. > 294247-01-10T04:00:54.776Z is one milli past the maximum allowed. > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/beam/pull/22347#issuecomment-1197298700>, or > unsubscribe > <https://github.com/notifications/unsubscribe-auth/AFAYJVJHUZBMZCS7K5BE7C3VWGHWVANCNFSM54BIPKIA> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
