[GitHub] incubator-beam pull request: [BEAM-117] Fix bug in PipelineOptions...

2016-05-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/355 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[jira] [Commented] (BEAM-117) Implement the API for Static Display Metadata

2016-05-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294621#comment-15294621 ] ASF GitHub Bot commented on BEAM-117: - Github user asfgit closed the pull request at:

[1/2] incubator-beam git commit: Fix bug in PipelineOptions DisplayData serialization

2016-05-20 Thread bchambers
Repository: incubator-beam Updated Branches: refs/heads/master 4755c5a78 -> bb086b8d3 Fix bug in PipelineOptions DisplayData serialization PipelineOptions has been improved to generate display data to be consumed by a runner and used for display. However, there was a bug in the

[2/2] incubator-beam git commit: This closes #355

2016-05-20 Thread bchambers
This closes #355 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bb086b8d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bb086b8d Diff:

[GitHub] incubator-beam pull request: Code cleanup in BigQueryIO

2016-05-20 Thread peihe
GitHub user peihe opened a pull request: https://github.com/apache/incubator-beam/pull/369 Code cleanup in BigQueryIO 1. Use toJsonString() to avoid try catch 2. add getTableWithDefaultProject in BigQuery.Write You can merge this pull request into a Git repository by running:

Jenkins build is back to stable : beam_PostCommit_RunnableOnService_GoogleCloudDataflow #387

2016-05-20 Thread Apache Jenkins Server
See

[GitHub] incubator-beam pull request: Add BoundedReader APIs for expressing...

2016-05-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/353 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: This closes #353

2016-05-20 Thread dhalperi
This closes #353 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4755c5a7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4755c5a7 Diff:

[1/2] incubator-beam git commit: BoundedReader: add getSplitPoints{Consumed, Remaining}

2016-05-20 Thread dhalperi
Repository: incubator-beam Updated Branches: refs/heads/master efd1f95b6 -> 4755c5a78 BoundedReader: add getSplitPoints{Consumed,Remaining} And implement and test it for common sources OffsetBasedReader: test limited parallelism signals AvroSource: rewrite to support remaining parallelism

[1/2] incubator-beam git commit: Closes #352

2016-05-20 Thread dhalperi
Repository: incubator-beam Updated Branches: refs/heads/master d7952959f -> efd1f95b6 Closes #352 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/efd1f95b Tree:

[2/2] incubator-beam git commit: [BEAM-48] Add BigQueryTornadoes integration test

2016-05-20 Thread dhalperi
[BEAM-48] Add BigQueryTornadoes integration test Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/89d20a2d Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/89d20a2d Diff:

[GitHub] incubator-beam pull request: Remove CustomSourcesTest

2016-05-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/368 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[2/2] incubator-beam git commit: Closes #368

2016-05-20 Thread dhalperi
Closes #368 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/d7952959 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/d7952959 Diff:

[GitHub] incubator-beam pull request: [Ignore this PR, trigger tests in a c...

2016-05-20 Thread peihe
Github user peihe closed the pull request at: https://github.com/apache/incubator-beam/pull/358 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] incubator-beam pull request: [Ignore this PR, trigger test in a cl...

2016-05-20 Thread peihe
Github user peihe closed the pull request at: https://github.com/apache/incubator-beam/pull/359 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[jira] [Updated] (BEAM-298) Make TestPipeline implement the TestRule interface

2016-05-20 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-298: -- Assignee: Mark Thomas (was: Davor Bonaci) > Make TestPipeline implement the TestRule interface >

[jira] [Updated] (BEAM-298) Make TestPipeline implement the TestRule interface

2016-05-20 Thread Davor Bonaci (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davor Bonaci updated BEAM-298: -- Assignee: Thomas Groh (was: Mark Thomas) > Make TestPipeline implement the TestRule interface >

[jira] [Created] (BEAM-299) MergingActiveWindowSet does not preserve invariants across persist/restore

2016-05-20 Thread Mark Shields (JIRA)
Mark Shields created BEAM-299: - Summary: MergingActiveWindowSet does not preserve invariants across persist/restore Key: BEAM-299 URL: https://issues.apache.org/jira/browse/BEAM-299 Project: Beam

[GitHub] incubator-beam pull request: Remove TestIO, Use CountingInput in C...

2016-05-20 Thread tgroh
GitHub user tgroh opened a pull request: https://github.com/apache/incubator-beam/pull/368 Remove TestIO, Use CountingInput in CustomSourcesTest Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [ ] Make sure the PR title

[GitHub] incubator-beam pull request: [BEAM-53] PubSub: build fixups

2016-05-20 Thread dhalperi
GitHub user dhalperi opened a pull request: https://github.com/apache/incubator-beam/pull/367 [BEAM-53] PubSub: build fixups * Always RunWith on a unit test * Mockito bugfix for Java 9 You can merge this pull request into a Git repository by running: $ git pull

Jenkins build is unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #386

2016-05-20 Thread Apache Jenkins Server
See

[jira] [Closed] (BEAM-222) TestCounterSource does not implement checkpointing contract

2016-05-20 Thread Mark Shields (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Shields closed BEAM-222. - Resolution: Fixed > TestCounterSource does not implement checkpointing contract >

Re: Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #384

2016-05-20 Thread Dan Halperin
We're looking into these tests that have been failing for two days now. Luckily, they're not on the pre-commit path so they're not blocking work. On Fri, May 20, 2016 at 1:27 AM, Apache Jenkins Server < jenk...@builds.apache.org> wrote: > See < >

[jira] [Commented] (BEAM-286) Reorganize flink runner directories

2016-05-20 Thread Maximilian Michels (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293080#comment-15293080 ] Maximilian Michels commented on BEAM-286: - The runner is composed of two submodules for a good

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #384

2016-05-20 Thread Apache Jenkins Server
See

[jira] [Resolved] (BEAM-258) Execute selected RunnableOnService tests with Flink runner

2016-05-20 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek resolved BEAM-258. --- Resolution: Fixed Fix Version/s: 0.1.0-incubating > Execute selected RunnableOnService

[jira] [Resolved] (BEAM-270) Support Timestamps/Windows in Flink Batch

2016-05-20 Thread Aljoscha Krettek (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aljoscha Krettek resolved BEAM-270. --- Resolution: Fixed Fix Version/s: 0.1.0-incubating > Support Timestamps/Windows in Flink

[jira] [Commented] (BEAM-270) Support Timestamps/Windows in Flink Batch

2016-05-20 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292883#comment-15292883 ] ASF GitHub Bot commented on BEAM-270: - Github user asfgit closed the pull request at:

[04/14] incubator-beam git commit: [BEAM-270] Support Timestamps/Windows in Flink Batch

2016-05-20 Thread aljoscha
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/24bfca23/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkDoFnFunction.java -- diff --git

[05/14] incubator-beam git commit: [BEAM-270] Support Timestamps/Windows in Flink Batch

2016-05-20 Thread aljoscha
[BEAM-270] Support Timestamps/Windows in Flink Batch With this change we always use WindowedValue for the underlying Flink DataSets instead of just T. This allows us to support windowing as well. This changes also a lot of other stuff enabled by the above: - Use WindowedValue throughout - Add

[08/14] incubator-beam git commit: Configure RunnableOnService tests for Flink in batch mode

2016-05-20 Thread aljoscha
Configure RunnableOnService tests for Flink in batch mode Today Flink batch supports only global windows. This is a situation we intend our build to allow, eventually via JUnit category filtering. For now all the test classes that use non-global windows are excluded entirely via maven

[GitHub] incubator-beam pull request: [BEAM-270] Support Timestamps/Windows...

2016-05-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-beam/pull/343 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[07/14] incubator-beam git commit: Add TestFlinkPipelineRunner to FlinkRunnerRegistrar

2016-05-20 Thread aljoscha
Add TestFlinkPipelineRunner to FlinkRunnerRegistrar This makes the runner available for selection by integration tests. Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/58d66a34 Tree:

[12/14] incubator-beam git commit: Disable Flink streaming integration tests for now

2016-05-20 Thread aljoscha
Disable Flink streaming integration tests for now Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/2d71af71 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/2d71af71 Diff:

[03/14] incubator-beam git commit: [BEAM-270] Support Timestamps/Windows in Flink Batch

2016-05-20 Thread aljoscha
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/24bfca23/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkReduceFunction.java -- diff --git

[01/14] incubator-beam git commit: Add surefire plugin to java 8 example tests

2016-05-20 Thread aljoscha
Repository: incubator-beam Updated Branches: refs/heads/master dc98211cc -> af8f5935c Add surefire plugin to java 8 example tests Without this the test will just pick up whathever was last written to beamTestPipelineOptions. Project:

[14/14] incubator-beam git commit: This closes #343

2016-05-20 Thread aljoscha
This closes #343 Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/af8f5935 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/af8f5935 Diff:

[10/14] incubator-beam git commit: Add hamcrest dependency to Flink Runner

2016-05-20 Thread aljoscha
Add hamcrest dependency to Flink Runner Without it the RunnableOnService tests seem to not work Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/1664c96d Tree:

[06/14] incubator-beam git commit: Special casing job exec AssertionError in TestFlinkPipelineRunner

2016-05-20 Thread aljoscha
Special casing job exec AssertionError in TestFlinkPipelineRunner Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/af8e9887 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/af8e9887 Diff:

[13/14] incubator-beam git commit: Fix Dangling Flink DataSets

2016-05-20 Thread aljoscha
Fix Dangling Flink DataSets Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/26fa0b21 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/26fa0b21 Diff:

[02/14] incubator-beam git commit: [BEAM-270] Support Timestamps/Windows in Flink Batch

2016-05-20 Thread aljoscha
http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/24bfca23/runners/flink/runner/src/test/java/org/apache/beam/runners/flink/TfIdfITCase.java -- diff --git

[09/14] incubator-beam git commit: Remove unused threadCount from integration tests

2016-05-20 Thread aljoscha
Remove unused threadCount from integration tests Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/55f39bf7 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/55f39bf7 Diff:

[11/14] incubator-beam git commit: Fix faulty Flink Flatten when PCollectionList is empty

2016-05-20 Thread aljoscha
Fix faulty Flink Flatten when PCollectionList is empty Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/4e60a497 Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/4e60a497 Diff:

Jenkins build is still unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #383

2016-05-20 Thread Apache Jenkins Server
See

[GitHub] incubator-beam pull request: [Beam 287] Add new groupId

2016-05-20 Thread jbonofre
GitHub user jbonofre opened a pull request: https://github.com/apache/incubator-beam/pull/365 [Beam 287] Add new groupId Be sure to do all of the following to help us incorporate your contribution quickly and easily: - [X] Make sure the PR title is formatted like: