[
https://issues.apache.org/jira/browse/BEAM-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16081817#comment-16081817
]
Aljoscha Krettek commented on BEAM-2571:
----------------------------------------
What's the problem in Dataflow?
In the real world this virtually doesn't occur, IMHO. The reason we're catching
this here is because of the side inputs and the holding back of the main input.
Things go roughly like this:
- main-input elements arrive, {{DoFnOperator}} puts them in state and holds
the input watermark back according to their timestamp
- input watermark already goes to +Inf at this point
- the first side-input element arrives, we notice that we can process one of
the pushed-pack elements, the watermark is still "tightly held" by the other
pushed-back elements, that's why it is exactly at the window end for some
windows
> Flink ValidatesRunner failing CombineTest.testSlidingWindowsCombineWithContext
> ------------------------------------------------------------------------------
>
> Key: BEAM-2571
> URL: https://issues.apache.org/jira/browse/BEAM-2571
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Kenneth Knowles
> Assignee: Aljoscha Krettek
> Fix For: 2.1.0
>
>
> This appears to have been caused by https://github.com/apache/beam/pull/3429
> which fixes a couple errors in how trigger timers were processed / final
> panes labeled.
> I am investigating, considering roll back vs forward fix. Since it is an
> esoteric use case where I would advise users to use a stateful DoFn instead,
> I think the bug fixed probably outweighs the bug introduced. I would like to
> fix for 2.1.0 but will report back soon.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
