[
https://issues.apache.org/jira/browse/BEAM-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225226#comment-15225226
]
ASF GitHub Bot commented on BEAM-162:
-------------------------------------
GitHub user mshields822 opened a pull request:
https://github.com/apache/incubator-beam/pull/118
[BEAM-162] Sensible interaction of merging and closed triggers
Same bug, better approach.
- We always check for closed triggers on the post-merged window state.
- We are careful not to leak NEW windows.
- More unit tests.
- Along the way, discovered we were clearing trigger finished bits.
Since TriggerRunner does not currently have its own unit tests,
added new unit test in ReduceFnRunnerTest to hit that specific case.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mshields822/incubator-beam beam-162
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit 55c34a553264491bd42db4197793eff436858b73
Author: Mark Shields <markshie...@google.com>
Date: 2016-04-04T20:31:12Z
Clear finished bits from merged-away windows, not merge result window.
commit 69eb778b354cd63fb23a48789c93ed48c2a01961
Author: Mark Shields <markshie...@google.com>
Date: 2016-03-31T18:36:01Z
Check for closed windows post-merging rather than pre-merging.
Make sure we garbage collect NEW windows which end up being for closed
windows.
Add unit tests to confirm.
----
> assert fail using session windows
> ---------------------------------
>
> Key: BEAM-162
> URL: https://issues.apache.org/jira/browse/BEAM-162
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Mark Shields
> Assignee: Mark Shields
>
> java.lang.IllegalStateException: Window
> [2016-03-31T05:35:31.158Z..2016-03-31T06:05:31.158Z) should have been added
> at
> com.google.cloud.dataflow.sdk.repackaged.com.google.common.base.Preconditions.checkState(Preconditions.java:199)
> at
> com.google.cloud.dataflow.sdk.util.ReduceFnRunner.processElement(ReduceFnRunner.java:440)
> at
> com.google.cloud.dataflow.sdk.util.ReduceFnRunner.processElements(ReduceFnRunner.java:282)
> at
> com.google.cloud.dataflow.sdk.util.GroupAlsoByWindowViaWindowSetDoFn.processElement(GroupAlsoByWindowViaWindowSetDoFn.java:83)
> at
> com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49)
> at
> com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:138)
> at
> com.google.cloud.dataflow.sdk.util.LateDataDroppingDoFnRunner.processElement(LateDataDroppingDoFnRunner.java:67)
> at
> com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:191)
> at
> com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42)
> at
> com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47)
> at
> com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:53)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
