[5/6] beam git commit: Rename AfterFirst to AfterAny for consistency with Java.

2017-04-20 Thread robertwb
Rename AfterFirst to AfterAny for consistency with Java. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2c347192 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2c347192 Diff:

[1/2] beam git commit: [BEAM-662] Fix for allowing floating point periods in windows

2017-04-20 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 4e0c8333c -> 4e0d5f596 [BEAM-662] Fix for allowing floating point periods in windows Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/1bc1bdd3 Tree:

[2/2] beam git commit: Closes #2600

2017-04-20 Thread robertwb
Closes #2600 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4e0d5f59 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4e0d5f59 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/4e0d5f59 Branch: refs/heads/master

[GitHub] beam pull request #2595: [Beam-115] More complete translation of the graph t...

2017-04-19 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2595 [Beam-115] More complete translation of the graph through the Runner API Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[GitHub] beam pull request #2586: Remove vestigial Read and Write from core.py

2017-04-18 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2586 Remove vestigial Read and Write from core.py Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[GitHub] beam pull request #2552: Various cleanups

2017-04-17 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2552 Various cleanups I accumulated these while working on other code. You can merge this pull request into a Git repository by running: $ git pull https://github.com/robertwb/incubator-beam

[1/2] beam git commit: Ignore more python build artifacts.

2017-04-11 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 77712c936 -> a5a5bf946 Ignore more python build artifacts. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a1683578 Tree:

[2/2] beam git commit: Closes #2494

2017-04-11 Thread robertwb
Closes #2494 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a5a5bf94 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/a5a5bf94 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/a5a5bf94 Branch: refs/heads/master

[GitHub] beam pull request #2494: Ignore more python build artifacts.

2017-04-11 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2494 Ignore more python build artifacts. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[GitHub] beam pull request #2437: Use absolute import for dataflow iobase test.

2017-04-05 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2437 Use absolute import for dataflow iobase test. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[1/2] beam git commit: Closes #2435

2017-04-05 Thread robertwb
Repository: beam Updated Branches: refs/heads/master bc907c58b -> e32a0252f Closes #2435 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e32a0252 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e32a0252 Diff:

[2/2] beam git commit: Revert "Add ValueProvider class for FileBasedSource I/O Transforms"

2017-04-05 Thread robertwb
Revert "Add ValueProvider class for FileBasedSource I/O Transforms" This reverts commit 1e2168a127fb3047fb15d231a001bbf951892e11. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3eef246f Tree:

[1/2] beam git commit: Closes #2432

2017-04-05 Thread robertwb
Repository: beam Updated Branches: refs/heads/master e2a2836ad -> 8e5cfdea9 Closes #2432 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/8e5cfdea Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/8e5cfdea Diff:

[2/2] beam git commit: Update refcounts after pipeline reconstruction.

2017-04-05 Thread robertwb
Update refcounts after pipeline reconstruction. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/54360814 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/54360814 Diff:

[GitHub] beam pull request #2432: Update refcounts after pipeline reconstruction.

2017-04-05 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2432 Update refcounts after pipeline reconstruction. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] beam git commit: Closes #2400

2017-04-04 Thread robertwb
Closes #2400 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/50fc63a9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/50fc63a9 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/50fc63a9 Branch: refs/heads/master

[1/2] beam git commit: Avoid Runner API translation of pipelines with PDone.

2017-04-04 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 22ff898a3 -> 50fc63a9b Avoid Runner API translation of pipelines with PDone. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a419 Tree:

[GitHub] beam pull request #2400: [BEAM-1843] Avoid Runner API translation of pipelin...

2017-04-01 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2400 [BEAM-1843] Avoid Runner API translation of pipelines with PDone. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR

[2/2] beam git commit: Fix side inputs on dataflow runner.

2017-03-31 Thread robertwb
Fix side inputs on dataflow runner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/07daf3a5 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/07daf3a5 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/07daf3a5

[1/2] beam git commit: Closes #2395

2017-03-31 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 60901f876 -> 03dce6dcc Closes #2395 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/03dce6dc Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/03dce6dc Diff:

[2/3] beam git commit: Only encode PCollection outputs in Runner API protos.

2017-03-31 Thread robertwb
Only encode PCollection outputs in Runner API protos. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c9ff44af Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c9ff44af Diff:

[3/3] beam git commit: Closes #2396

2017-03-31 Thread robertwb
Closes #2396 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/60901f87 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/60901f87 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/60901f87 Branch: refs/heads/master

[1/3] beam git commit: Ensure transforms are picklable before materializing to protos.

2017-03-31 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 07499824b -> 60901f876 Ensure transforms are picklable before materializing to protos. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bd40c5a2 Tree:

[GitHub] beam pull request #2396: [BEAM-1843] Ensure transforms are picklable before ...

2017-03-31 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2396 [BEAM-1843] Ensure transforms are picklable before materializing to protos. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make

[GitHub] beam pull request #2395: [BEAM-1853] Fix side inputs on dataflow runner.

2017-03-31 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2395 [BEAM-1853] Fix side inputs on dataflow runner. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[1/2] beam git commit: Closes #2388

2017-03-31 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 132d3c5f6 -> affb926cc Closes #2388 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/affb926c Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/affb926c Diff:

[2/2] beam git commit: Change side inputs to be references rather than full PValues.

2017-03-31 Thread robertwb
). -self._view_cache = {} def _current_transform(self): """Returns the transform currently on the top of the stack.""" @@ -271,8 +268,8 @@ class Pipeline(object): result.producer = current # TODO(robertwb): Multi-input, multi-output inference

[GitHub] beam pull request #2388: Change side inputs to be references rather than ful...

2017-03-30 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2388 Change side inputs to be references rather than full PValues. This is more consistent with the Runner API's structure. Be sure to do all of the following to help us incorporate your

[1/2] beam git commit: Translate pipeline graph to and from Runner API protos.

2017-03-30 Thread robertwb
ne_context +context = pipeline_context.PipelineContext(proto.components) +p.transforms_stack = [ +context.transforms.get_by_id(proto.root_transform_id)] +# TODO(robertwb): These are only needed to continue construction. Omit? +p.applied_labels = set([ +t.unique_name for t in proto.co

[GitHub] beam pull request #2318: Add link to contributions guides to github README

2017-03-24 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/beam/pull/2318 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[1/2] beam git commit: Allow customizing the GCE metadata service URL.

2017-03-24 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 9b8e16fca -> c80ef1837 Allow customizing the GCE metadata service URL. The goal here is to allow a user to customize where a job finds the metadata service; it would also be possible to do this programmatically (eg expose a variable), but

[2/2] beam git commit: Closes #2319

2017-03-24 Thread robertwb
Closes #2319 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c80ef183 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c80ef183 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c80ef183 Branch: refs/heads/master

[GitHub] beam pull request #2318: Add link to contributions guides to github README

2017-03-24 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2318 Add link to contributions guides to github README Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] beam pull request #2296: [BEAM-115] Translate pipeline graph to and from Run...

2017-03-22 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2296 [BEAM-115] Translate pipeline graph to and from Runner API protos. There are some caveates: * Specific known transforms, with their payloads, are not yet translated. * Side inputs

[GitHub] beam-site pull request #176: Update quickstart-java.md

2017-03-21 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/beam-site/pull/176 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[2/3] beam git commit: Fix bad test obscured by falsely passing assert_that.

2017-03-21 Thread robertwb
Fix bad test obscured by falsely passing assert_that. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/47da10cf Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/47da10cf Diff:

[1/3] beam git commit: Closes #2279

2017-03-21 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 030528f3d -> bea4f5aec Closes #2279 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bea4f5ae Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/bea4f5ae Diff:

[3/3] beam git commit: [BEAM-1768] Fix assert_that for empty inputs

2017-03-21 Thread robertwb
[BEAM-1768] Fix assert_that for empty inputs Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c39b02d9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c39b02d9 Diff:

[GitHub] beam pull request #2279: [BEAM-1768] Fix assert_that for empty inputs

2017-03-20 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2279 [BEAM-1768] Fix assert_that for empty inputs Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted

[GitHub] beam-site pull request #176: Update quickstart-java.md

2017-03-13 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam-site/pull/176 Update quickstart-java.md I had to make this change for a clean build to work with the 0.6.0 source release. You can merge this pull request into a Git repository by running: $ git pull

[GitHub] beam pull request #2233: Better error on missing gcp dependencies.

2017-03-13 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2233 Better error on missing gcp dependencies. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[7/9] beam git commit: Runner API encoding of WindowFns.

2017-03-09 Thread robertwb
Runner API encoding of WindowFns. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/aad32b7a Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/aad32b7a Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/aad32b7a Branch:

[8/9] beam git commit: Move pipeline context and add more tests.

2017-03-09 Thread robertwb
+ unittest.main() http://git-wip-us.apache.org/repos/asf/beam/blob/deff128f/sdks/python/apache_beam/transforms/core.py -- diff --git a/sdks/python/apache_beam/transforms/core.py b/sdks/python/apache_beam/transforms/core.py index 1fc63b2..3251671

[3/9] beam git commit: Auto-generated runner api proto bindings.

2017-03-09 Thread robertwb
http://git-wip-us.apache.org/repos/asf/beam/blob/3bb125e1/sdks/python/apache_beam/runners/api/beam_runner_api_pb2.py -- diff --git a/sdks/python/apache_beam/runners/api/beam_runner_api_pb2.py

[1/9] beam git commit: Runner API context helper classes.

2017-03-09 Thread robertwb
Repository: beam Updated Branches: refs/heads/master f13a84d67 -> 2c2424cb4 Runner API context helper classes. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/bc76a186 Tree:

[6/9] beam git commit: Runner API translation of triggers and windowing strategies.

2017-03-09 Thread robertwb
turn beam_runner_api_pb2.WindowingStrategy( +window_fn=self.windowfn.to_runner_api(context), +# TODO(robertwb): Prohibit implicit multi-level merging. +merge_status=(beam_runner_api_pb2.NEEDS_MERGE + if self.windowfn.is_merging() +

[2/9] beam git commit: Auto-generated runner api proto bindings.

2017-03-09 Thread robertwb
http://git-wip-us.apache.org/repos/asf/beam/blob/3bb125e1/sdks/python/run_pylint.sh -- diff --git a/sdks/python/run_pylint.sh b/sdks/python/run_pylint.sh index afc5fb4..5e63856 100755 --- a/sdks/python/run_pylint.sh +++

[9/9] beam git commit: Closes #2190

2017-03-09 Thread robertwb
Closes #2190 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2c2424cb Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/2c2424cb Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/2c2424cb Branch: refs/heads/master

[5/9] beam git commit: Add license to new files.

2017-03-09 Thread robertwb
Add license to new files. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b2da21e2 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b2da21e2 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b2da21e2 Branch:

[GitHub] beam pull request #2214: Ensure that assert_that takes a PCollection as its ...

2017-03-09 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2214 Ensure that assert_that takes a PCollection as its first argument. This can avoid (silent) error such as with test_pipeline.TestPipeline() as p: bad_value = "Cr

[GitHub] beam pull request #2191: Fix typo in proto: widow -> window.

2017-03-07 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2191 Fix typo in proto: widow -> window. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted l

[GitHub] beam pull request #2190: [BEAM-115] Runner API representation of windowing s...

2017-03-07 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2190 [BEAM-115] Runner API representation of windowing strategies for Python Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure

[1/2] beam git commit: Updates Python SDK source API so that sources can report limited parallelism signals.

2017-03-02 Thread robertwb
Repository: beam Updated Branches: refs/heads/master de9e8528c -> 8f5f19d11 Updates Python SDK source API so that sources can report limited parallelism signals. With this update Python BoundedSource/RangeTracker API can report consumed and remaining number of split points while performing

[1/2] beam git commit: Inline rather than reference FunctionSpecs.

2017-03-01 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 79b1395c2 -> d84b06791 Inline rather than reference FunctionSpecs. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/d390406e Tree:

[2/2] beam git commit: Closes #2131

2017-03-01 Thread robertwb
Closes #2131 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/d84b0679 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/d84b0679 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/d84b0679 Branch: refs/heads/master

[1/3] beam git commit: Include cython tests in presubmits for linux platform

2017-02-28 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 3f4e008bf -> 352af855e Include cython tests in presubmits for linux platform Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2a03dd4e Tree:

[3/3] beam git commit: Closes #2128

2017-02-28 Thread robertwb
Closes #2128 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/352af855 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/352af855 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/352af855 Branch: refs/heads/master

[2/3] beam git commit: Update output stream cython declaration

2017-02-28 Thread robertwb
Update output stream cython declaration Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/b498e115 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/b498e115 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/b498e115

[GitHub] beam pull request #2131: [BEAM-115] Inline rather than reference FunctionSpe...

2017-02-28 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2131 [BEAM-115] Inline rather than reference FunctionSpecs. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] beam git commit: Make side inputs a map, rather than embedding the name in the message.

2017-02-28 Thread robertwb
Make side inputs a map, rather than embedding the name in the message. The "local" name only make sense in context. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/2d59b10f Tree:

[1/2] beam git commit: Closes #2124

2017-02-28 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 61e31e622 -> 3f4e008bf Closes #2124 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3f4e008b Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3f4e008b Diff:

[GitHub] beam pull request #2124: Make side inputs a map, rather than embedding the n...

2017-02-27 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2124 Make side inputs a map, rather than embedding the name in the message. The "local" name only make sense in context. Be sure to do all of the following to help us incorp

[1/3] beam git commit: Make access_pattern a URN with params.

2017-02-27 Thread robertwb
Repository: beam Updated Branches: refs/heads/master c8dfb851c -> 16736a6b6 Make access_pattern a URN with params. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/a0dcd3f9 Tree:

[3/3] beam git commit: Closes #2106

2017-02-27 Thread robertwb
Closes #2106 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/16736a6b Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/16736a6b Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/16736a6b Branch: refs/heads/master

[2/3] beam git commit: Clarify side input name and remove redundant pcollection reference.

2017-02-27 Thread robertwb
Clarify side input name and remove redundant pcollection reference. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c04e8abd Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c04e8abd Diff:

[GitHub] beam pull request #2106: [BEAM-115] More Runner API refinements.

2017-02-24 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/2106 [BEAM-115] More Runner API refinements. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like

[2/2] beam git commit: Attach original traceback in direct runner failure.

2017-02-10 Thread robertwb
Attach original traceback in direct runner failure. This will make it much easier to debug pipelines. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/5fba260d Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/5fba260d

[1/2] beam git commit: Closes #1970

2017-02-10 Thread robertwb
Repository: beam Updated Branches: refs/heads/master effca63f9 -> 51ef0009a Closes #1970 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/51ef0009 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/51ef0009 Diff:

[2/2] beam git commit: Closes #1959

2017-02-10 Thread robertwb
Closes #1959 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/caaa64dc Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/caaa64dc Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/caaa64dc Branch: refs/heads/master

[1/2] beam git commit: BEAM-1443: Preserve window duplication on window assignment.

2017-02-10 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 9c4a784bb -> caaa64dc6 BEAM-1443: Preserve window duplication on window assignment. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/71264671 Tree:

[2/2] beam git commit: Closes #1969

2017-02-09 Thread robertwb
Closes #1969 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/6ab60d89 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/6ab60d89 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/6ab60d89 Branch: refs/heads/master

[GitHub] beam pull request #1970: Attach original exception in direct runner failure.

2017-02-09 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/1970 Attach original exception in direct runner failure. This will make it much easier to debug pipelines. Be sure to do all of the following to help us incorporate your contribution quickly

[1/2] beam git commit: [BEAM-1450] Fix NewDoFn handling of window explosion.

2017-02-09 Thread robertwb
Repository: beam Updated Branches: refs/heads/master c26d5827b -> cd6802bec [BEAM-1450] Fix NewDoFn handling of window explosion. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/37b6fb1a Tree:

[GitHub] beam pull request #1965: [BEAM-1450] Fix NewDoFn handling of window explosio...

2017-02-09 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/1965 [BEAM-1450] Fix NewDoFn handling of window explosion. Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[2/2] beam git commit: Closes #1845

2017-02-06 Thread robertwb
Closes #1845 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/f90558c3 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/f90558c3 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/f90558c3 Branch: refs/heads/master

[1/2] beam git commit: Add snippet for reading from compressed text sources

2017-02-06 Thread robertwb
Repository: beam Updated Branches: refs/heads/master a26fd1ff3 -> f90558c34 Add snippet for reading from compressed text sources Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/98e513bc Tree:

[1/2] beam git commit: Reduce cost of sample combine test

2017-02-01 Thread robertwb
Repository: beam Updated Branches: refs/heads/master 2689ca43c -> 420a71860 Reduce cost of sample combine test There's no need to run this test 300 times, which takes tens of seconds. Also split global and per-key sampling into separate tests. Project:

[1/3] beam git commit: Implement combiner lifting for direct runner.

2017-01-31 Thread robertwb
Repository: beam Updated Branches: refs/heads/master b80aac5e3 -> e9cd41165 Implement combiner lifting for direct runner. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/0de5cf87 Tree:

[GitHub] beam pull request #1882: Combiner lifting

2017-01-31 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/1882 Combiner lifting Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title is formatted like: `[BEAM

[GitHub] beam-site pull request #131: Clarify maturity of various runners.

2017-01-27 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam-site/pull/131 Clarify maturity of various runners. R: @francesperry You can merge this pull request into a Git repository by running: $ git pull https://github.com/robertwb/incubator-beam-site patch-2

[1/3] beam git commit: Updating dataflow client protos to add new metrics.

2017-01-27 Thread robertwb
Repository: beam Updated Branches: refs/heads/python-sdk 3d6f20d67 -> 52d97e2fc http://git-wip-us.apache.org/repos/asf/beam/blob/901a14c4/sdks/python/apache_beam/internal/clients/dataflow/dataflow_v1b3_messages.py -- diff

[3/3] beam git commit: Closes #1857

2017-01-27 Thread robertwb
Closes #1857 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/52d97e2f Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/52d97e2f Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/52d97e2f Branch:

[2/3] beam git commit: Updating dataflow client protos to add new metrics.

2017-01-27 Thread robertwb
Updating dataflow client protos to add new metrics. In order to use counter structured names, and add new metrics (e.g. Distributions), we need to update the dataflow client protocol buffers. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[1/2] beam git commit: Refactoring metrics infrastructure

2017-01-26 Thread robertwb
mport SideOutputValue from apache_beam.transforms import core from apache_beam.transforms import window @@ -69,7 +70,8 @@ class DoFnRunner(Receiver): logging_context=None, # Preferred alternative to context # TODO(robertwb): Remove once all runners

[2/2] beam git commit: Closes #1835

2017-01-26 Thread robertwb
Closes #1835 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/3d6f20d6 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/3d6f20d6 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/3d6f20d6 Branch:

[1/2] beam git commit: Fix read/write display data

2017-01-26 Thread robertwb
Repository: beam Updated Branches: refs/heads/python-sdk c6420df97 -> e3849af8c Fix read/write display data Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e4eda3c3 Tree:

[2/2] beam git commit: Closes #1776

2017-01-26 Thread robertwb
Closes #1776 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/e3849af8 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/e3849af8 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/e3849af8 Branch:

[GitHub] beam pull request #1837: [BEAM-1218 ] Remove dataflow_test.py

2017-01-25 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/beam/pull/1837 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[2/2] beam git commit: Closes #1837

2017-01-25 Thread robertwb
Closes #1837 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/c6420df9 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/c6420df9 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/c6420df9 Branch:

[1/2] beam git commit: Closes #1844

2017-01-25 Thread robertwb
Repository: beam Updated Branches: refs/heads/python-sdk 592422059 -> 4e1028b3d Closes #1844 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/4e1028b3 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/4e1028b3 Diff:

[2/2] beam git commit: Use a temp directory for requirements cache in test_with_requirements_file

2017-01-25 Thread robertwb
Use a temp directory for requirements cache in test_with_requirements_file The test fails if there are leftover files in the default folder for requirements cache either from earlier tests, or from the previous workspaces. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[GitHub] beam pull request #1811: Cleanup tests in pipeline_test.

2017-01-25 Thread robertwb
Github user robertwb closed the pull request at: https://github.com/apache/beam/pull/1811 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[2/2] beam git commit: Cleanup tests in pipeline_test.

2017-01-25 Thread robertwb
Cleanup tests in pipeline_test. Notably, the runner_name parameter has been obsolete since the removal of DiskCachedRunnerPipelineTest and is an inferior version of what TestPipeline provides. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit:

[GitHub] beam pull request #1837: [BEAM-1218 ] Remove dataflow_test.py

2017-01-24 Thread robertwb
GitHub user robertwb opened a pull request: https://github.com/apache/beam/pull/1837 [BEAM-1218 ] Remove dataflow_test.py This reverts commit 96fcc7d31c2540f867c3a73903c2aa99183a6b8b. Be sure to do all of the following to help us incorporate your contribution quickly

[2/2] beam git commit: Closes #1836

2017-01-24 Thread robertwb
Closes #1836 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/9540cf17 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/9540cf17 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/9540cf17 Branch:

[2/2] beam git commit: Closes #1820

2017-01-24 Thread robertwb
Closes #1820 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/43cb4d70 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/43cb4d70 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/43cb4d70 Branch:

[1/2] beam git commit: Removes Dataflow native text source and sink from Beam SDK.

2017-01-24 Thread robertwb
Repository: beam Updated Branches: refs/heads/python-sdk f9831236c -> 43cb4d709 Removes Dataflow native text source and sink from Beam SDK. Users should be using Beam text source and sink available in module 'textio.py' instead of this. Also removes Dataflow native file source/sink that is

[2/2] beam git commit: Closes #1832

2017-01-24 Thread robertwb
Closes #1832 Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/f9831236 Tree: http://git-wip-us.apache.org/repos/asf/beam/tree/f9831236 Diff: http://git-wip-us.apache.org/repos/asf/beam/diff/f9831236 Branch:

[1/2] beam git commit: Fix case where side inputs may be an iterable rather than a list.

2017-01-24 Thread robertwb
Repository: beam Updated Branches: refs/heads/python-sdk d0dc1f375 -> f9831236c Fix case where side inputs may be an iterable rather than a list. Project: http://git-wip-us.apache.org/repos/asf/beam/repo Commit: http://git-wip-us.apache.org/repos/asf/beam/commit/9052366f Tree:

[1/2] beam git commit: Revert "Remove dataflow_test.py"

2017-01-24 Thread robertwb
lf): -return TestPipeline('DirectRunner') +return beam.Pipeline('DirectRunner') def run_windowed_side_inputs(self, elements, main_window_fn, side_window_fn=None, @@ -128,209 +125,6 @@ class SideInputsTest(unittest.TestCase): (11, {'k11': 'v11

<    13   14   15   16   17   18   19   >