[ https://issues.apache.org/jira/browse/BEAM-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15668726#comment-15668726 ]
ASF GitHub Bot commented on BEAM-986: ------------------------------------- GitHub user scwhittle opened a pull request: https://github.com/apache/incubator-beam/pull/1366 [BEAM-986] Improve ReduceFnRunner prefetching Hi @kennknowles , can you please take a look? This refactoring ensures that for batches of elements or timers we have at most 2 fetches, one to prefetch the should fire bits and one to fetch the actual firing state. The current behavior can end up with 2 fetches per window triggered. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scwhittle/incubator-beam reduce_fn_prefetching Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1366.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1366 ---- commit 597af025865c6ae2208fc94ea8864491cd84ff37 Author: Sam Whittle <samu...@google.com> Date: 2016-11-10T20:59:49Z Improvements to ReduceFnRunner prefetching: - add prefetch* methods for prefetching state matching existing methods - replace onTimer with batched onTimers method to allow prefetching across timers - prefetch triggers in processElements commit 8f11421019e2acd603546071ae0d8383143a33be Author: Sam Whittle <samu...@google.com> Date: 2016-11-15T22:55:38Z fix annotation ---- > ReduceFnRunner doesn't batch prefetching pane firings > ----------------------------------------------------- > > Key: BEAM-986 > URL: https://issues.apache.org/jira/browse/BEAM-986 > Project: Beam > Issue Type: Bug > Components: runner-core > Reporter: Sam Whittle > Assignee: Sam Whittle > Original Estimate: 24h > Remaining Estimate: 24h > > Specifically > - in ProcessElements, if there are multiple windows to consider each is > processed sequentially with sequential state fetches instead of a bulk > prefetch > - onTimer method doesn't evaluate multiple timers at a time meaning that if > multiple timers are fired at once each is processed sequentially without > batched prefetching -- This message was sent by Atlassian JIRA (v6.3.4#6332)