[GitHub] incubator-beam pull request #1564: [Beam-986] Fix LateDataDroppingDoFnRunner...
GitHub user scwhittle opened a pull request: https://github.com/apache/incubator-beam/pull/1564 [Beam-986] Fix LateDataDroppingDoFnRunner to not increment the counter on reiter⦠Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM-] Description of pull request` - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable Travis-CI on your fork and ensure the whole test matrix passes). - [ ] Replace `` in the title with the actual Jira issue number, if there is one. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt). --- â¦ation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scwhittle/incubator-beam fix_late_data_counter Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1564.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1564 commit ea6b96684865d1d061c57a753050e191061bab1f Author: Sam Whittle <samu...@google.com> Date: 2016-12-09T20:43:04Z Fix LateDataDroppingDoFnRunner to not increment the counter on reiteration. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1519: [BEAM-986] Improvements to ReduceFnRunner...
GitHub user scwhittle opened a pull request: https://github.com/apache/incubator-beam/pull/1519 [BEAM-986] Improvements to ReduceFnRunner prefetching - add prefetch* methods for prefetching state matching existing methods - replace onTimer with batched onTimers method to allow prefetching across timers - remove deprecated TimerCallback usage - prefetch triggers in processElements You can merge this pull request into a Git repository by running: $ git pull https://github.com/scwhittle/incubator-beam reduce_fn_prefetching_no_sdk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1519.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1519 commit 5085c77736a314519c6dcdb4c4c2c21b425b9668 Author: Sam Whittle <samu...@google.com> Date: 2016-11-10T20:59:49Z Improvements to ReduceFnRunner prefetching: - add prefetch* methods for prefetching state matching existing methods - replace onTimer with batched onTimers method to allow prefetching across timers - prefetch triggers in processElements Additionally remove deprecated TimerCallback usage --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1493: [BEAM-986] Deprecate TimerCallback and In...
GitHub user scwhittle opened a pull request: https://github.com/apache/incubator-beam/pull/1493 [BEAM-986] Deprecate TimerCallback and InMemoryTimerInternals methods using it. Deprecate TimerCallback and InMemoryTimerInternals methods using it. Instead separate advancing watermarks and removing eligible timers. SDK changes necessary for improving ReduceFnRunner prefetching. Hi @kennknowles , can you please take a look? This is the sdk changes separated out from PR adding prefetching. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scwhittle/incubator-beam rm_timer_callback Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1493.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1493 commit 550c6b05b426714f291174323366f82d34afeab3 Author: Sam Whittle <samu...@google.com> Date: 2016-12-02T06:51:37Z Deprecate TimerCallback and InMemoryTimerInternals methods using it. Instead separate advancing watermarks and removing eligible timers. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-beam pull request #1366: [BEAM-986] Improve ReduceFnRunner prefetc...
GitHub user scwhittle opened a pull request: https://github.com/apache/incubator-beam/pull/1366 [BEAM-986] Improve ReduceFnRunner prefetching Hi @kennknowles , can you please take a look? This refactoring ensures that for batches of elements or timers we have at most 2 fetches, one to prefetch the should fire bits and one to fetch the actual firing state. The current behavior can end up with 2 fetches per window triggered. You can merge this pull request into a Git repository by running: $ git pull https://github.com/scwhittle/incubator-beam reduce_fn_prefetching Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-beam/pull/1366.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1366 commit 597af025865c6ae2208fc94ea8864491cd84ff37 Author: Sam Whittle <samu...@google.com> Date: 2016-11-10T20:59:49Z Improvements to ReduceFnRunner prefetching: - add prefetch* methods for prefetching state matching existing methods - replace onTimer with batched onTimers method to allow prefetching across timers - prefetch triggers in processElements commit 8f11421019e2acd603546071ae0d8383143a33be Author: Sam Whittle <samu...@google.com> Date: 2016-11-15T22:55:38Z fix annotation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
