[
https://issues.apache.org/jira/browse/BEAM-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16002039#comment-16002039
]
Jingsong Lee commented on BEAM-2140:
------------------------------------
First, {{SplittableParDo}} should not wrap {{StatefulDoFnRunner}}.
Second, {{SplittableParDo}} use {{PROCESSING_TIME}} to continue processing. And
it also sets watermark holds which will affect the sending of the output
watermark. (see {{DoFnOperator.processWatermark1()}}).
When {{BoundedSourceWrapper}} is over, it will emit a Long.MAX_VALUE watermark,
but the {{SplittableParDo}} may be not over yet. (depends on system time) So no
one can send watermark to the downstream.
Last, {{StreamTask}} will shutdown when there are no inputs and invoke
{{timerService.quiesceAndAwaitPending}}. (see {{StreamTask.invoke()}} in Flink)
It will shutdown TimeService and invoke all task in TimeService and reject the
new registration. So it will break the continue processing of
{{SplittableParDo}}.
[~aljoscha] Is that right? Please correct me if I wrong.
> Fix SplittableDoFn ValidatesRunner tests in FlinkRunner
> -------------------------------------------------------
>
> Key: BEAM-2140
> URL: https://issues.apache.org/jira/browse/BEAM-2140
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Aljoscha Krettek
> Assignee: Aljoscha Krettek
>
> As discovered as part of BEAM-1763, there is a failing SDF test. We disabled
> the tests to unblock the open PR for BEAM-1763.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
