date:20160930

[jira] [Commented] (BEAM-290) Integrate Apache Beam into Apache Zeppelin

2016-09-30 Thread Mahmoud Elgamal (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535524#comment-15535524
 ] 

Mahmoud Elgamal commented on BEAM-290:
--

The PR of beam interpreter is accepted, and I'm working on improving it.
https://github.com/apache/zeppelin/pull/1334

> Integrate Apache Beam into Apache Zeppelin
> --
>
> Key: BEAM-290
> URL: https://issues.apache.org/jira/browse/BEAM-290
> Project: Beam
>  Issue Type: Wish
>  Components: runner-ideas
>Reporter: Ismaël Mejía
>Priority: Minor
>
> This task is to follow the issue of supporting Apache Beam as an execution 
> environment for Apache Zeppelin notebooks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (BEAM-695) DisplayData for PipelineOptions fails to correctly toString array types

2016-09-30 Thread Luke Cwik (JIRA)

Luke Cwik created BEAM-695:
--

 Summary: DisplayData for PipelineOptions fails to correctly 
toString array types
 Key: BEAM-695
 URL: https://issues.apache.org/jira/browse/BEAM-695
 Project: Beam
  Issue Type: Bug
  Components: sdk-java-core
Reporter: Luke Cwik
Assignee: Scott Wegner
Priority: Minor


For array types in Java, toString produces an uninformative message like
[Ljava.lang.String;@fc258b1

You need to check to see if its an array type and call Arrays.toString(array).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #996: [BEAM-310] Add RootTransformEvaluatorFacto...

2016-09-30 Thread tgroh

Github user tgroh closed the pull request at:

https://github.com/apache/incubator-beam/pull/996


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-310) Exercise splitIntoBundles/generateInitialSplits in the Direct Runner

2016-09-30 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536420#comment-15536420
 ] 

ASF GitHub Bot commented on BEAM-310:
-

Github user tgroh closed the pull request at:

https://github.com/apache/incubator-beam/pull/996


> Exercise splitIntoBundles/generateInitialSplits in the Direct Runner
> 
>
> Key: BEAM-310
> URL: https://issues.apache.org/jira/browse/BEAM-310
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-direct
>Reporter: Thomas Groh
>Assignee: Thomas Groh
>
> BoundedSource#splitIntoBundles and UnboundedSource#generateInitialSplits are 
> the methods by which sources can be accessed in parallel. Exercising these 
> methods allows reads (and all transforms downstream) to be executed in 
> parallel both pre and post a GroupByKey



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-beam pull request #1034: [BEAM-695] PipelineOptions display data n...

2016-09-30 Thread swegner

GitHub user swegner opened a pull request:

https://github.com/apache/incubator-beam/pull/1034

[BEAM-695] PipelineOptions display data needs to handle array types

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

PipelineOptions generates display data for arbitrary option types
using #toString(). For array types, this gives an message like
[Ljava.lang.String;@fc25934b]. Instead, we detect array types
and use Arrays.deepString to build a string based on array values.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/swegner/incubator-beam 
displaydata-array-string

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1034.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1034


commit ca12648f5ce0b744c30cfa6bbcd486519ae1bf95
Author: Scott Wegner 
Date:   2016-09-30T17:05:00Z

PipelineOptions display data needs to handle array types

PipelineOptions generates display data for arbitrary option types
using #toString(). For array types, this gives an message like
[Ljava.lang.String;@fc25934b]. Instead, we detect array types
and use Arrays.deepString to build a string based on array values.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-695) DisplayData for PipelineOptions fails to correctly toString array types

2016-09-30 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536490#comment-15536490
 ] 

ASF GitHub Bot commented on BEAM-695:
-

GitHub user swegner opened a pull request:

https://github.com/apache/incubator-beam/pull/1034

[BEAM-695] PipelineOptions display data needs to handle array types

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---

PipelineOptions generates display data for arbitrary option types
using #toString(). For array types, this gives an message like
[Ljava.lang.String;@fc25934b]. Instead, we detect array types
and use Arrays.deepString to build a string based on array values.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/swegner/incubator-beam 
displaydata-array-string

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1034.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1034


commit ca12648f5ce0b744c30cfa6bbcd486519ae1bf95
Author: Scott Wegner 
Date:   2016-09-30T17:05:00Z

PipelineOptions display data needs to handle array types

PipelineOptions generates display data for arbitrary option types
using #toString(). For array types, this gives an message like
[Ljava.lang.String;@fc25934b]. Instead, we detect array types
and use Arrays.deepString to build a string based on array values.




> DisplayData for PipelineOptions fails to correctly toString array types
> ---
>
> Key: BEAM-695
> URL: https://issues.apache.org/jira/browse/BEAM-695
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-java-core
>Reporter: Luke Cwik
>Assignee: Scott Wegner
>Priority: Minor
>
> For array types in Java, toString produces an uninformative message like  
> [Ljava.lang.String;@fc258b1
> You need to check to see if its an array type and call Arrays.toString(array).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] incubator-beam git commit: Closes #1023

2016-09-30 Thread bchambers

Closes #1023


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/0d229f5a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/0d229f5a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/0d229f5a

Branch: refs/heads/master
Commit: 0d229f5a69059024377202e09eab7d2a2f332a84
Parents: e7684b7 425c778
Author: bchambers 
Authored: Fri Sep 30 11:12:39 2016 -0700
Committer: bchambers 
Committed: Fri Sep 30 11:12:39 2016 -0700

--
 .../beam/runners/core/BatchTimerInternals.java  | 140 ---
 .../GroupAlsoByWindowsViaOutputBufferDoFn.java  |   7 +-
 .../beam/runners/core/ReduceFnRunner.java   |   3 +-
 .../runners/core/BatchTimerInternalsTest.java   | 118 -
 .../beam/runners/core/ReduceFnTester.java   | 248 ++-
 .../sdk/util/state/InMemoryTimerInternals.java  | 235 ++
 .../util/state/TestInMemoryStateInternals.java  |  61 +
 .../beam/sdk/util/state/TimerCallback.java  |  35 +++
 .../org/apache/beam/sdk/util/TriggerTester.java | 206 +--
 .../util/state/InMemoryTimerInternalsTest.java  | 116 +
 10 files changed, 482 insertions(+), 687 deletions(-)
--

[GitHub] incubator-beam pull request #1023: Deduplicates the 3 in-memory TimerInterna...

2016-09-30 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1023


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[1/2] incubator-beam git commit: Dedups TimerInternal implementations

2016-09-30 Thread bchambers

Repository: incubator-beam
Updated Branches:
  refs/heads/master e7684b79b -> 0d229f5a6


Dedups TimerInternal implementations

Factors InMemoryTimerInternals out of ReduceFnTester

Uses InMemoryTimerInternals instead of BatchTimerInternals and 
TriggerTester.TestTimerInternals

Previously, there were 3 implementations:
TestTimerInternals in ReduceFnTester
TestTimerInternals in TriggerTester (these two were nearly identical)
BatchTimerInternals (it was a subset of the above)
There were also 2 copies of TestInMemoryStateInternals.

This change deduplicates and reorganizes them:

1. Deduplicates the TestInMemoryStateInternals.
2. Factors out the common timer stuff into InMemoryTimerInternals.
3. TriggerTester's implementation of TestTimerInternals used to
  (unnecessarily) access TestInMemoryStateInternals, presumably due to
  copy-paste. Now it uses the regular InMemoryTimerInternals.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/425c7781
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/425c7781
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/425c7781

Branch: refs/heads/master
Commit: 425c77818fc7bbd919fd1ed10a1c3a10b3bfb920
Parents: e7684b7
Author: Eugene Kirpichov 
Authored: Wed Sep 28 19:19:12 2016 -0700
Committer: bchambers 
Committed: Fri Sep 30 11:11:27 2016 -0700

--
 .../beam/runners/core/BatchTimerInternals.java  | 140 ---
 .../GroupAlsoByWindowsViaOutputBufferDoFn.java  |   7 +-
 .../beam/runners/core/ReduceFnRunner.java   |   3 +-
 .../runners/core/BatchTimerInternalsTest.java   | 118 -
 .../beam/runners/core/ReduceFnTester.java   | 248 ++-
 .../sdk/util/state/InMemoryTimerInternals.java  | 235 ++
 .../util/state/TestInMemoryStateInternals.java  |  61 +
 .../beam/sdk/util/state/TimerCallback.java  |  35 +++
 .../org/apache/beam/sdk/util/TriggerTester.java | 206 +--
 .../util/state/InMemoryTimerInternalsTest.java  | 116 +
 10 files changed, 482 insertions(+), 687 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/425c7781/runners/core-java/src/main/java/org/apache/beam/runners/core/BatchTimerInternals.java
--
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/BatchTimerInternals.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/BatchTimerInternals.java
deleted file mode 100644
index 829dbde..000
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/BatchTimerInternals.java
+++ /dev/null
@@ -1,140 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.beam.runners.core;
-
-import static com.google.common.base.Preconditions.checkState;
-
-import com.google.common.base.MoreObjects;
-import java.util.HashSet;
-import java.util.PriorityQueue;
-import java.util.Set;
-import javax.annotation.Nullable;
-import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
-import org.apache.beam.sdk.util.TimeDomain;
-import org.apache.beam.sdk.util.TimerInternals;
-
-import org.joda.time.Instant;
-
-/**
- * TimerInternals that uses priority queues to manage the timers that are 
ready to fire.
- */
-public class BatchTimerInternals implements TimerInternals {
-  /** Set of timers that are scheduled used for deduplicating timers. */
-  private Set existingTimers = new HashSet<>();
-
-  // Keep these queues separate so we can advance over them separately.
-  private PriorityQueue watermarkTimers = new PriorityQueue<>(11);
-  private PriorityQueue processingTimers = new PriorityQueue<>(11);
-
-  private Instant inputWatermarkTime;
-  private Instant processingTime;
-
-  private PriorityQueue queue(TimeDomain domain) {
-return TimeDomain.EVENT_TIME.equals(domain) ? watermarkTimers : 
processingTimers;
-  }
-
-  public BatchTimerInternals(Instant processingTime) {
-this.processingTime = processingTime;
-this.inputWatermarkTime = Bou

Build failed in Jenkins: beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform #1427

2016-09-30 Thread Apache Jenkins Server

See 


--
[INFO] 
[INFO] 
[INFO] Building Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform 
0.3.0-incubating-SNAPSHOT
[INFO] 
[INFO] Downloading: 
https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-codec-http2/maven-metadata.xml
[INFO] Downloading: 
https://repository.apache.org/content/repositories/snapshots/io/netty/netty-codec-http2/maven-metadata.xml
[INFO] Downloaded: 
https://oss.sonatype.org/content/repositories/snapshots/io/netty/netty-codec-http2/maven-metadata.xml
 (405 B at 1.4 KB/sec)
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 15 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 15 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Surefire report directory: 

[INFO] Using configured provider 
org.apache.maven.surefire.junitcore.JUnitCoreProvider
[INFO] parallel='none', perCoreThreadCount=true, threadCount=0, 
useUnlimitedThreads=false, threadCountSuites=0, threadCountClasses=0, 
threadCountMethods=0, parallelOptimized=true

---
 T E S T S
---
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryTableRowIteratorTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elaps

Build failed in Jenkins: beam_PostCommit_MavenVerify #1427

2016-09-30 Thread Apache Jenkins Server

See 

--
[...truncated 3040 lines...]
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 15 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 15 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.17:check (default) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Starting audit...
Audit done.
[INFO] 
[INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Surefire report directory: 

[INFO] Using configured provider 
org.apache.maven.surefire.junitcore.JUnitCoreProvider
[INFO] parallel='none', perCoreThreadCount=true, threadCount=0, 
useUnlimitedThreads=false, threadCountSuites=0, threadCountClasses=0, 
threadCountMethods=0, parallelOptimized=true

---
 T E S T S
---
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryTableRowIteratorTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.622 sec - in 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTableRowIteratorTest
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryAvroUtilsTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.534 sec - in 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryAvroUtilsTest
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryTableInserterTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.837 sec - in 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryTableInserterTest
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOTest
Tests run: 60, Failures: 0, Errors: 0, Skipped: 4, Time elapsed: 3.629 sec - in 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryIOTest
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtilTest
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.139 sec - in 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryUtilTest
Running org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImplTest
Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 sec - in 
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImplTest
Running org.apache.beam.sdk.io.gcp.bigtable.BigtableIOTest
Tests run: 25, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.111 sec - in 
org.apache.beam.sdk.io.gcp.bigtable.BigtableIOTest
Running org.apache.beam.sdk.io.gcp.datastore.DatastoreV1Test
Tests run: 43, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.383 sec - in 
org.apache.beam.sdk.io.gcp.datastore.DatastoreV1Test

Results :

Tests run: 158, Failures: 0, Errors: 0, Skipped: 4

[JENKINS] Recording test results
[INFO] 
[INFO] --- maven-jar-plugin:2.5:jar (default-jar) @ 
beam-sdks-java-io-google-cloud-platform ---
[INFO] Building jar:

[1/2] incubator-beam git commit: Enable EncodabilityEnforcement by default

2016-09-30 Thread bchambers

Repository: incubator-beam
Updated Branches:
  refs/heads/master 0d229f5a6 -> 7d46698f2


Enable EncodabilityEnforcement by default

This ensures that all elements, rather than only non-null elements, will
have their encodability checked.

Also, only encode elements in EncodabilityEnforcement once.

Instead of checking that all input elements are encodable, ensure all
elements produced by a PTransform can be encoded with the provided
coder. This reduces the number of duplicate checks performed and enables
EncodabilityEnforcement to be attached to Read PTransforms to ensure
that provided coders can encode all elements output by a source.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/cdd88a98
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/cdd88a98
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/cdd88a98

Branch: refs/heads/master
Commit: cdd88a983fd2937110d52ea4661e256c8f4422f6
Parents: 0d229f5
Author: Thomas Groh 
Authored: Tue Aug 23 09:37:07 2016 -0700
Committer: bchambers 
Committed: Fri Sep 30 11:23:42 2016 -0700

--
 .../beam/runners/direct/DirectOptions.java  |  13 +-
 .../beam/runners/direct/DirectRunner.java   |  16 ++-
 .../direct/EncodabilityEnforcementFactory.java  |  50 +---
 .../beam/runners/direct/DirectRunnerTest.java   |  66 ++
 .../EncodabilityEnforcementFactoryTest.java | 122 +++
 5 files changed, 215 insertions(+), 52 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/cdd88a98/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectOptions.java
--
diff --git 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectOptions.java
 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectOptions.java
index 798fda4..89e1bb8 100644
--- 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectOptions.java
+++ 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectOptions.java
@@ -49,10 +49,17 @@ public interface DirectOptions extends PipelineOptions, 
ApplicationNameOptions {
 
   @Default.Boolean(true)
   @Description(
-  "Controls whether the runner should ensure that all of the elements of 
every "
+  "Controls whether the DirectRunner should ensure that all of the 
elements of every "
   + "PCollection are not mutated. PTransforms are not permitted to 
mutate input elements "
   + "at any point, or output elements after they are output.")
-  boolean isTestImmutability();
+  boolean isEnforceImmutability();
 
-  void setTestImmutability(boolean test);
+  void setEnforceImmutability(boolean test);
+
+  @Default.Boolean(true)
+  @Description(
+  "Controls whether the DirectRunner should ensure that all of the 
elements of every "
+  + "PCollection are encodable. All elements in a PCollection must be 
encodable.")
+  boolean isEnforceEncodability();
+  void setEnforceEncodability(boolean test);
 }

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/cdd88a98/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectRunner.java
--
diff --git 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectRunner.java
 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectRunner.java
index a3d20f6..2ec4f08 100644
--- 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectRunner.java
+++ 
b/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DirectRunner.java
@@ -39,6 +39,7 @@ import org.apache.beam.sdk.Pipeline;
 import org.apache.beam.sdk.Pipeline.PipelineExecutionException;
 import org.apache.beam.sdk.PipelineResult;
 import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.io.Read;
 import org.apache.beam.sdk.io.Write;
 import org.apache.beam.sdk.options.PipelineOptions;
 import org.apache.beam.sdk.runners.PipelineRunner;
@@ -284,21 +285,32 @@ public class DirectRunner
 Collection parDoEnforcements = 
createParDoEnforcements(options);
 enforcements.put(ParDo.Bound.class, parDoEnforcements);
 enforcements.put(ParDo.BoundMulti.class, parDoEnforcements);
+if (options.isEnforceEncodability()) {
+  enforcements.put(
+  Read.Unbounded.class,
+  
ImmutableSet.of(EncodabilityEnforcementFactory.create()));
+  enforcements.put(
+  Read.Bounded.class,
+  
ImmutableSet.of(EncodabilityEnforcementFactory.create()));
+}
 return enforcements.build();
   }
 
   private Collection createParDoEnforcements(
   DirectOptions options) {
 ImmutableList.Builder enforce

[2/2] incubator-beam git commit: Closes #864

2016-09-30 Thread bchambers

Closes #864


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/7d46698f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/7d46698f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/7d46698f

Branch: refs/heads/master
Commit: 7d46698f2878e41369077c75f9347aafbeff0c5a
Parents: 0d229f5 cdd88a9
Author: bchambers 
Authored: Fri Sep 30 11:24:22 2016 -0700
Committer: bchambers 
Committed: Fri Sep 30 11:24:22 2016 -0700

--
 .../beam/runners/direct/DirectOptions.java  |  13 +-
 .../beam/runners/direct/DirectRunner.java   |  16 ++-
 .../direct/EncodabilityEnforcementFactory.java  |  50 +---
 .../beam/runners/direct/DirectRunnerTest.java   |  66 ++
 .../EncodabilityEnforcementFactoryTest.java | 122 +++
 5 files changed, 215 insertions(+), 52 deletions(-)
--

[GitHub] incubator-beam pull request #864: Enable EncodabilityEnforcement by Default

2016-09-30 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/864


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build is back to normal : beam_PostCommit_MavenVerify » Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform #1428

2016-09-30 Thread Apache Jenkins Server

See

Jenkins build is back to normal : beam_PostCommit_MavenVerify #1428

2016-09-30 Thread Apache Jenkins Server

See

[1/2] incubator-beam git commit: Forward port PR-411 and PR-420 from Dataflow

2016-09-30 Thread lcwik

Repository: incubator-beam
Updated Branches:
  refs/heads/master 7d46698f2 -> b237e2f05


Forward port PR-411 and PR-420 from Dataflow


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/8b4550d2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/8b4550d2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/8b4550d2

Branch: refs/heads/master
Commit: 8b4550d27920b53d04291dc383f28c9f7f77ca32
Parents: 7d46698
Author: Pei He 
Authored: Thu Sep 29 15:13:28 2016 -0700
Committer: Luke Cwik 
Committed: Fri Sep 30 14:13:46 2016 -0700

--
 .../gcp/bigquery/BigQueryTableRowIterator.java  | 114 ++-
 .../bigquery/BigQueryTableRowIteratorTest.java  | 143 +++
 2 files changed, 190 insertions(+), 67 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/8b4550d2/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableRowIterator.java
--
diff --git 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableRowIterator.java
 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableRowIterator.java
index 0ee01d9..64b1dc6 100644
--- 
a/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableRowIterator.java
+++ 
b/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryTableRowIterator.java
@@ -28,7 +28,6 @@ import com.google.api.client.util.ClassInfo;
 import com.google.api.client.util.Data;
 import com.google.api.client.util.Sleeper;
 import com.google.api.services.bigquery.Bigquery;
-import com.google.api.services.bigquery.Bigquery.Jobs.Insert;
 import com.google.api.services.bigquery.model.Dataset;
 import com.google.api.services.bigquery.model.DatasetReference;
 import com.google.api.services.bigquery.model.ErrorProto;
@@ -36,6 +35,7 @@ import com.google.api.services.bigquery.model.Job;
 import com.google.api.services.bigquery.model.JobConfiguration;
 import com.google.api.services.bigquery.model.JobConfigurationQuery;
 import com.google.api.services.bigquery.model.JobReference;
+import com.google.api.services.bigquery.model.JobStatistics;
 import com.google.api.services.bigquery.model.JobStatus;
 import com.google.api.services.bigquery.model.Table;
 import com.google.api.services.bigquery.model.TableCell;
@@ -137,16 +137,7 @@ class BigQueryTableRowIterator implements AutoCloseable {
   ref = executeQueryAndWaitForCompletion();
 }
 // Get table schema.
-Bigquery.Tables.Get get =
-client.tables().get(ref.getProjectId(), ref.getDatasetId(), 
ref.getTableId());
-
-Table table =
-executeWithBackOff(
-get,
-"Error opening BigQuery table  %s of dataset %s  : {}",
-ref.getTableId(),
-ref.getDatasetId());
-schema = table.getSchema();
+schema = getTable(ref).getSchema();
   }
 
   public boolean advance() throws IOException, InterruptedException {
@@ -168,12 +159,11 @@ class BigQueryTableRowIterator implements AutoCloseable {
 list.setPageToken(pageToken);
   }
 
-  TableDataList result =
-  executeWithBackOff(
-  list,
-  "Error reading from BigQuery table %s of dataset %s : {}",
-  ref.getTableId(),
-  ref.getDatasetId());
+  TableDataList result = executeWithBackOff(
+  list,
+  String.format(
+  "Error reading from BigQuery table %s of dataset %s.",
+  ref.getTableId(), ref.getDatasetId()));
 
   pageToken = result.getPageToken();
   iteratorOverCurrentBatch =
@@ -332,19 +322,36 @@ class BigQueryTableRowIterator implements AutoCloseable {
 return row;
   }
 
+  // Get the BiqQuery table.
+  private Table getTable(TableReference ref) throws IOException, 
InterruptedException {
+Bigquery.Tables.Get get =
+client.tables().get(ref.getProjectId(), ref.getDatasetId(), 
ref.getTableId());
+
+return executeWithBackOff(
+get,
+String.format(
+"Error opening BigQuery table %s of dataset %s.",
+ref.getTableId(),
+ref.getDatasetId()));
+  }
+
   // Create a new BigQuery dataset
-  private void createDataset(String datasetId) throws IOException, 
InterruptedException {
+  private void createDataset(String datasetId, @Nullable String location)
+  throws IOException, InterruptedException {
 Dataset dataset = new Dataset();
 DatasetReference reference = new DatasetReference();
 reference.setProjectId(projectId);
 reference.setDatasetId(datasetId);
 dataset.setDatasetRefer

[2/2] incubator-beam git commit: Forward port PR-411 and PR-420 from Dataflow

2016-09-30 Thread lcwik

Forward port PR-411 and PR-420 from Dataflow

This closes #1032


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/b237e2f0
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/b237e2f0
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/b237e2f0

Branch: refs/heads/master
Commit: b237e2f052b2a80c56b0914133a58ed5fdb9dbcf
Parents: 7d46698 8b4550d
Author: Luke Cwik 
Authored: Fri Sep 30 14:14:18 2016 -0700
Committer: Luke Cwik 
Committed: Fri Sep 30 14:14:18 2016 -0700

--
 .../gcp/bigquery/BigQueryTableRowIterator.java  | 114 ++-
 .../bigquery/BigQueryTableRowIteratorTest.java  | 143 +++
 2 files changed, 190 insertions(+), 67 deletions(-)
--

[GitHub] incubator-beam pull request #1032: Forward port PR-411 and PR-420 from Dataf...

2016-09-30 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1032


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Build failed in Jenkins: beam_PostCommit_MavenVerify » Apache Beam :: Parent #1430

2016-09-30 Thread Apache Jenkins Server

See 


--
Established TCP socket on 40446
maven32-agent.jar already up to date
maven32-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; 
support was removed in 8.0
<===[JENKINS REMOTING CAPACITY]===>   channel started
Executing Maven:  -B -f 

 
-Dmaven.repo.local=
 -B -e -P release -DrepoToken=YZWbqQQnANEGMBgOFvXZEOr7etawgMcOQ clean install 
coveralls:report -DskipITs=false -DintegrationTestPipelineOptions=[ 
"--project=apache-beam-testing", 
"--tempRoot=gs://temp-storage-for-end-to-end-tests", 
"--runner=org.apache.beam.runners.dataflow.testing.TestDataflowRunner" ]
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] 
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Beam :: Parent
[INFO] Apache Beam :: SDKs
[INFO] Apache Beam :: SDKs :: Java
[INFO] Apache Beam :: SDKs :: Java :: Build Tools
[INFO] Apache Beam :: SDKs :: Java :: Core
[INFO] Apache Beam :: Runners
[INFO] Apache Beam :: Runners :: Core Java
[INFO] Apache Beam :: Runners :: Direct Java
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow
[INFO] Apache Beam :: SDKs :: Java :: IO
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform
[INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB
[INFO] Apache Beam :: SDKs :: Java :: Extensions
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library
[INFO] Apache Beam :: SDKs :: Java :: Microbenchmarks
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests
[INFO] Apache Beam :: Runners :: Flink
[INFO] Apache Beam :: Runners :: Flink :: Core
[INFO] Apache Beam :: Runners :: Flink :: Examples
[INFO] Apache Beam :: Runners :: Spark
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples
[INFO] Apache Beam :: Examples
[INFO] Apache Beam :: Examples :: Java
[INFO] Apache Beam :: Examples :: Java 8
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
no dependency information available
[WARNING] Failed to retrieve plugin descriptor for 
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be 
resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in 
https://repo.maven.apache.org/maven2 was cached in the local repository, 
resolution will not be reattempted until the update interval of central has 
elapsed or updates are forced
[INFO] 
[INFO] 
[INFO] Building Apache Beam :: Parent 0.3.0-incubating-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ beam-parent ---
[INFO] Deleting 

[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce) @ beam-parent ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ beam-parent ---
[INFO] 
[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ 
beam-parent ---
[INFO] 
[INFO] --- maven-dependency-plugin:2.10:analyze-only (default) @ beam-parent ---
[INFO] Skipping pom project
[INFO] 
[INFO] --- apache-rat-plugin:0.11:check (default) @ beam-parent ---
[INFO] 51 implicit excludes (use -debug for more details).
[INFO] Exclude: **/target/**/*
[INFO] Exclude: **/dependency-reduced-pom.xml
[INFO] Exclude: .github/**/*
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/package-list
[INFO] Exclude: **/user.avsc
[INFO] Exclude: **/test/resources/**/*.txt
[INFO] Exclude: **/test/**/.placeholder
[INFO] Exclude: .repository/**/*
[INFO] Exclude: **/.checkstyle
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.factorypath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**/*
[INFO] 1227 resources included (use -debug for more details)
[INFO] Rat check: Summary of files. Unapproved: 1 unknown: 1 generated: 0 
approved: 1222 licence.

Build failed in Jenkins: beam_PostCommit_MavenVerify #1430

2016-09-30 Thread Apache Jenkins Server

See 

Changes:

[lcwik] Forward port PR-411 and PR-420 from Dataflow

--
[EnvInject] - Mask passwords passed as build parameters.
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on beam1 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/incubator-beam.git # 
 > timeout=10
Fetching upstream changes from https://github.com/apache/incubator-beam.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://github.com/apache/incubator-beam.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision b237e2f052b2a80c56b0914133a58ed5fdb9dbcf 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f b237e2f052b2a80c56b0914133a58ed5fdb9dbcf
 > git rev-list 7d46698f2878e41369077c75f9347aafbeff0c5a # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
Modules changed, recalculating dependency graph
Established TCP socket on 40446
maven32-agent.jar already up to date
maven32-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
[beam_PostCommit_MavenVerify] $ /home/jenkins/tools/java/latest1.8/bin/java 
-Xmx2g -Xms256m -XX:MaxPermSize=512m -cp 
/home/jenkins/jenkins-slave/maven32-agent.jar:/home/jenkins/tools/maven/apache-maven-3.3.3/boot/plexus-classworlds-2.5.2.jar:/home/jenkins/tools/maven/apache-maven-3.3.3/conf/logging
 jenkins.maven3.agent.Maven32Main /home/jenkins/tools/maven/apache-maven-3.3.3 
/home/jenkins/jenkins-slave/slave.jar 
/home/jenkins/jenkins-slave/maven32-interceptor.jar 
/home/jenkins/jenkins-slave/maven3-interceptor-commons.jar 40446
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; 
support was removed in 8.0
<===[JENKINS REMOTING CAPACITY]===>   channel started
Executing Maven:  -B -f 
 
-Dmaven.repo.local=
 -B -e -P release -DrepoToken= clean install coveralls:report 
-DskipITs=false -DintegrationTestPipelineOptions=[ 
"--project=apache-beam-testing", 
"--tempRoot=gs://temp-storage-for-end-to-end-tests", 
"--runner=org.apache.beam.runners.dataflow.testing.TestDataflowRunner" ]
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] 
[INFO] Reactor Build Order:
[INFO] 
[INFO] Apache Beam :: Parent
[INFO] Apache Beam :: SDKs
[INFO] Apache Beam :: SDKs :: Java
[INFO] Apache Beam :: SDKs :: Java :: Build Tools
[INFO] Apache Beam :: SDKs :: Java :: Core
[INFO] Apache Beam :: Runners
[INFO] Apache Beam :: Runners :: Core Java
[INFO] Apache Beam :: Runners :: Direct Java
[INFO] Apache Beam :: Runners :: Google Cloud Dataflow
[INFO] Apache Beam :: SDKs :: Java :: IO
[INFO] Apache Beam :: SDKs :: Java :: IO :: Google Cloud Platform
[INFO] Apache Beam :: SDKs :: Java :: IO :: HDFS
[INFO] Apache Beam :: SDKs :: Java :: IO :: JMS
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kafka
[INFO] Apache Beam :: SDKs :: Java :: IO :: Kinesis
[INFO] Apache Beam :: SDKs :: Java :: IO :: MongoDB
[INFO] Apache Beam :: SDKs :: Java :: Extensions
[INFO] Apache Beam :: SDKs :: Java :: Extensions :: Join library
[INFO] Apache Beam :: SDKs :: Java :: Microbenchmarks
[INFO] Apache Beam :: SDKs :: Java :: Java 8 Tests
[INFO] Apache Beam :: Runners :: Flink
[INFO] Apache Beam :: Runners :: Flink :: Core
[INFO] Apache Beam :: Runners :: Flink :: Examples
[INFO] Apache Beam :: Runners :: Spark
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Starter
[INFO] Apache Beam :: SDKs :: Java :: Maven Archetypes :: Examples
[INFO] Apache Beam :: Examples
[INFO] Apache Beam :: Examples :: Java
[INFO] Apache Beam :: Examples :: Java 8
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
no dependency information available
[WARNING] Failed to retrieve plugin descriptor for 
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be 
resolved: Failure to find org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in 
https://repo.maven.apache.org/maven2 was cached in the local repository, 
resolution will not be reattempted until the update

[GitHub] incubator-beam pull request #1035: [BEAM] Making sure that GcsBufferedReader...

2016-09-30 Thread pabloem

GitHub user pabloem opened a pull request:

https://github.com/apache/incubator-beam/pull/1035

[BEAM] Making sure that GcsBufferedReader implements the iterator protocol

The change adds only the `__iter__` and `next` functions to the 
`GcsBufferedReader` class, so that it implements the iterator protocol in 
Python.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/pabloem/incubator-beam python-sdk

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1035.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1035






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Jenkins build became unstable: beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1244

2016-09-30 Thread Apache Jenkins Server

See

[GitHub] incubator-beam pull request #1036: Introduces the Rebundle transform

2016-09-30 Thread jkff

GitHub user jkff opened a pull request:

https://github.com/apache/incubator-beam/pull/1036

Introduces the Rebundle transform

It's similar to Reshuffle in that it prevents fusion of the surrounding 
transforms, however while Reshuffle
requires the input collection to be KVs, Rebundle efficiently generates 
sufficiently unique keys itself.

Also uses it in Datastore. The transform will be useful in JdbcIO.

(I tried adapting it to also support the case of fixed number of bundles, 
as in the Write transform, but this has hairy semantics in the unbounded case, 
so I decided not to do it)

R: @bjchambers 
CC: @jbonofre @dhalperi 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jkff/incubator-beam reparallelize

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-beam/pull/1036.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1036


commit 58dc834c88b2a1249645f307276c97c9b9fa0e78
Author: Eugene Kirpichov 
Date:   2016-09-30T18:18:38Z

Introduces the Rebundle transform

It's similar to Reshuffle in that it prevents fusion
of the surrounding transforms, however while Reshuffle
requires the input collection to be KVs, Rebundle
efficiently generates sufficiently unique keys itself.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (BEAM-696) Side-Inputs non-deterministic with merging main-input windows

2016-09-30 Thread Ben Chambers (JIRA)

Ben Chambers created BEAM-696:
-

 Summary: Side-Inputs non-deterministic with merging main-input 
windows
 Key: BEAM-696
 URL: https://issues.apache.org/jira/browse/BEAM-696
 Project: Beam
  Issue Type: Bug
  Components: beam-model
Reporter: Ben Chambers
Assignee: Frances Perry


Side-Inputs are non-deterministic for several reasons:
1. Because they depend on triggering of the side-input (this is acceptable 
because triggers are by their nature non-deterministic).
2. They depend on the current state of the main-input window in order to lookup 
the side-input. This means that with merging
3. Any runner optimizations that affect when the side-input is looked up may 
cause problems with either or both of these.

This issue focuses on #2 -- the non-determinism of side-inputs that execute 
within a Merging WindowFn.

Possible solution would be to defer running anything that looks up the 
side-input until we need to extract an output, and using the main-window at 
that point. Specifically, if the main-window is a MergingWindowFn, don't 
execute any kind of pre-combine, instead buffer all the inputs and combine 
later.

This could still run into some non-determinism if there are triggers 
controlling when we extract output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] incubator-beam git commit: Set transform names in DisplayDataEvaluator

2016-09-30 Thread bchambers

Repository: incubator-beam
Updated Branches:
  refs/heads/master b237e2f05 -> bc80ee342


Set transform names in DisplayDataEvaluator

If a transform name is unset, a runner may infer the name from
the class simple name, which fails with anonymous classes. In order
to be more generically useful, we set transform names within the
DisplayDataEvaluator.

Handle anonymous classes in DataflowUnboundedReadFromBoundedSource.getKind()

Correctly implement display data for DataflowRunner Read transforms

The DataflowRunner will replace usages of the Read transform with
interally-wrapped implementations for streaming and bounded cases.
We were not properly passing display data between these usages.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/3e7c396a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/3e7c396a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/3e7c396a

Branch: refs/heads/master
Commit: 3e7c396a09965ad688fc9b584c57cb1807360391
Parents: b237e2f
Author: Scott Wegner 
Authored: Thu Sep 29 14:43:14 2016 -0700
Committer: bchambers 
Committed: Fri Sep 30 16:12:14 2016 -0700

--
 .../beam/runners/dataflow/DataflowRunner.java   |  8 ++
 .../DataflowUnboundedReadFromBoundedSource.java | 16 +++-
 ...aflowUnboundedReadFromBoundedSourceTest.java | 82 
 .../sdk/io/BoundedReadFromUnboundedSource.java  |  6 ++
 .../java/org/apache/beam/sdk/io/ReadTest.java   | 56 -
 .../display/DisplayDataEvaluator.java   |  7 +-
 6 files changed, 168 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/3e7c396a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
--
diff --git 
a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
 
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
index 54c95a7..ceaf6a0 100644
--- 
a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
+++ 
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
@@ -132,6 +132,7 @@ import org.apache.beam.sdk.transforms.SerializableFunction;
 import org.apache.beam.sdk.transforms.View;
 import org.apache.beam.sdk.transforms.View.CreatePCollectionView;
 import org.apache.beam.sdk.transforms.WithKeys;
+import org.apache.beam.sdk.transforms.display.DisplayData;
 import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
 import org.apache.beam.sdk.transforms.windowing.GlobalWindow;
 import org.apache.beam.sdk.transforms.windowing.GlobalWindows;
@@ -2258,6 +2259,13 @@ public class DataflowRunner extends 
PipelineRunner {
 return 
ValueWithRecordId.ValueWithRecordIdCoder.of(source.getDefaultOutputCoder());
   }
 
+  @Override
+  public void populateDisplayData(DisplayData.Builder builder) {
+super.populateDisplayData(builder);
+builder.add(DisplayData.item("source", source.getClass()));
+builder.include(source);
+  }
+
   public UnboundedSource getSource() {
 return source;
   }

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/3e7c396a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/internal/DataflowUnboundedReadFromBoundedSource.java
--
diff --git 
a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/internal/DataflowUnboundedReadFromBoundedSource.java
 
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/internal/DataflowUnboundedReadFromBoundedSource.java
index 866da13..e4257d1 100644
--- 
a/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/internal/DataflowUnboundedReadFromBoundedSource.java
+++ 
b/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/internal/DataflowUnboundedReadFromBoundedSource.java
@@ -105,7 +105,14 @@ public class DataflowUnboundedReadFromBoundedSource 
extends PTransform 
extends PTransform(boundedSource.getDefaultOutputCoder());
 }
 
+@Override
+public void populateDisplayData(DisplayData.Builder builder) {
+  super.populateDisplayData(builder);
+  builder.add(DisplayData.item("source", boundedSource.getClass()));
+  builder.include(boundedSource);
+}
+
 @VisibleForTesting
 static class Checkpoint implements UnboundedSource.CheckpointMark {
   private final @Nullable List> residualElements;

http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/3e7c396a

[2/2] incubator-beam git commit: Closes #1029

2016-09-30 Thread bchambers

Closes #1029


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/bc80ee34
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/bc80ee34
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/bc80ee34

Branch: refs/heads/master
Commit: bc80ee342749768bbfc2a9fda80a4b15680e5898
Parents: b237e2f 3e7c396
Author: bchambers 
Authored: Fri Sep 30 16:12:47 2016 -0700
Committer: bchambers 
Committed: Fri Sep 30 16:12:47 2016 -0700

--
 .../beam/runners/dataflow/DataflowRunner.java   |  8 ++
 .../DataflowUnboundedReadFromBoundedSource.java | 16 +++-
 ...aflowUnboundedReadFromBoundedSourceTest.java | 82 
 .../sdk/io/BoundedReadFromUnboundedSource.java  |  6 ++
 .../java/org/apache/beam/sdk/io/ReadTest.java   | 56 -
 .../display/DisplayDataEvaluator.java   |  7 +-
 6 files changed, 168 insertions(+), 7 deletions(-)
--

[GitHub] incubator-beam pull request #1029: [BEAM-691] Correctly implement display da...

2016-09-30 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1029


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (BEAM-691) AvroIO missing display data for Dataflow runner in streaming mode

2016-09-30 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537391#comment-15537391
 ] 

ASF GitHub Bot commented on BEAM-691:
-

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-beam/pull/1029


> AvroIO missing display data for Dataflow runner in streaming mode
> -
>
> Key: BEAM-691
> URL: https://issues.apache.org/jira/browse/BEAM-691
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>
> AvroIO's Read display data does not show up in Dataflow runner for streaming 
> pipelines. This is because the runner has a different graph translation for 
> when StreamingOptions.isStreaming() is true, and the display data for this 
> case is not implemented.
> We should add display data to the primitive transforms for this translation, 
> and also explore ways to better test display data integration for different 
> possible graph translations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-696) Side-Inputs non-deterministic with merging main-input windows

2016-09-30 Thread Ben Chambers (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537421#comment-15537421
 ] 

Ben Chambers commented on BEAM-696:
---

This is currently manifesting as the assertions in 
CombineTest#testSessionsCombineWithContext failing.

> Side-Inputs non-deterministic with merging main-input windows
> -
>
> Key: BEAM-696
> URL: https://issues.apache.org/jira/browse/BEAM-696
> Project: Beam
>  Issue Type: Bug
>  Components: beam-model
>Reporter: Ben Chambers
>Assignee: Frances Perry
>
> Side-Inputs are non-deterministic for several reasons:
> 1. Because they depend on triggering of the side-input (this is acceptable 
> because triggers are by their nature non-deterministic).
> 2. They depend on the current state of the main-input window in order to 
> lookup the side-input. This means that with merging
> 3. Any runner optimizations that affect when the side-input is looked up may 
> cause problems with either or both of these.
> This issue focuses on #2 -- the non-determinism of side-inputs that execute 
> within a Merging WindowFn.
> Possible solution would be to defer running anything that looks up the 
> side-input until we need to extract an output, and using the main-window at 
> that point. Specifically, if the main-window is a MergingWindowFn, don't 
> execute any kind of pre-combine, instead buffer all the inputs and combine 
> later.
> This could still run into some non-determinism if there are triggers 
> controlling when we extract output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (BEAM-696) Side-Inputs non-deterministic with merging main-input windows

2016-09-30 Thread Ben Chambers (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ben Chambers updated BEAM-696:
--
Assignee: Pei He  (was: Frances Perry)

> Side-Inputs non-deterministic with merging main-input windows
> -
>
> Key: BEAM-696
> URL: https://issues.apache.org/jira/browse/BEAM-696
> Project: Beam
>  Issue Type: Bug
>  Components: beam-model
>Reporter: Ben Chambers
>Assignee: Pei He
>
> Side-Inputs are non-deterministic for several reasons:
> 1. Because they depend on triggering of the side-input (this is acceptable 
> because triggers are by their nature non-deterministic).
> 2. They depend on the current state of the main-input window in order to 
> lookup the side-input. This means that with merging
> 3. Any runner optimizations that affect when the side-input is looked up may 
> cause problems with either or both of these.
> This issue focuses on #2 -- the non-determinism of side-inputs that execute 
> within a Merging WindowFn.
> Possible solution would be to defer running anything that looks up the 
> side-input until we need to extract an output, and using the main-window at 
> that point. Specifically, if the main-window is a MergingWindowFn, don't 
> execute any kind of pre-combine, instead buffer all the inputs and combine 
> later.
> This could still run into some non-determinism if there are triggers 
> controlling when we extract output.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] incubator-beam git commit: Use keyword arguments in fnc calls

2016-09-30 Thread robertwb

Repository: incubator-beam
Updated Branches:
  refs/heads/python-sdk dc92438fa -> 731a77152


Use keyword arguments in fnc calls


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/f94eb53b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/f94eb53b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/f94eb53b

Branch: refs/heads/python-sdk
Commit: f94eb53b8b1fbf405d6fa62baeb9b9ee9090c1de
Parents: 5e87980
Author: Maria Garcia Herrero 
Authored: Fri Sep 30 10:41:38 2016 -0700
Committer: Robert Bradshaw 
Committed: Fri Sep 30 16:48:05 2016 -0700

--
 .../apache_beam/utils/annotations_test.py   | 51 
 1 file changed, 32 insertions(+), 19 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/f94eb53b/sdks/python/apache_beam/utils/annotations_test.py
--
diff --git a/sdks/python/apache_beam/utils/annotations_test.py 
b/sdks/python/apache_beam/utils/annotations_test.py
index af89590..854098c 100644
--- a/sdks/python/apache_beam/utils/annotations_test.py
+++ b/sdks/python/apache_beam/utils/annotations_test.py
@@ -30,10 +30,12 @@ class AnnotationTests(unittest.TestCase):
   def fnc_test_deprecated_with_since_current():
 return 'lol'
   fnc_test_deprecated_with_since_current()
-  self.check_annotation(w, 1, DeprecationWarning,
-'fnc_test_deprecated_with_since_current',
-'deprecated',
-[('since', True), ('instead', True)])
+  self.check_annotation(warning=w, warning_size=1,
+warning_type=DeprecationWarning,
+fnc_name='fnc_test_deprecated_with_since_current',
+annotation_type='deprecated',
+label_check_list=[('since', True),
+  ('instead', True)])
 
   def test_deprecated_without_current(self):
 with warnings.catch_warnings(record=True) as w:
@@ -41,9 +43,12 @@ class AnnotationTests(unittest.TestCase):
   def fnc_test_deprecated_without_current():
 return 'lol'
   fnc_test_deprecated_without_current()
-  self.check_annotation(w, 1, DeprecationWarning,
-'fnc_test_deprecated_without_current', 
'deprecated',
-[('since', True), ('instead', False)])
+  self.check_annotation(warning=w, warning_size=1,
+warning_type=DeprecationWarning,
+fnc_name='fnc_test_deprecated_without_current',
+annotation_type='deprecated',
+label_check_list=[('since', True),
+  ('instead', False)])
 
   def test_deprecated_without_since_should_fail(self):
 with warnings.catch_warnings(record=True) as w:
@@ -61,9 +66,11 @@ class AnnotationTests(unittest.TestCase):
   def fnc_test_experimental_with_current():
 return 'lol'
   fnc_test_experimental_with_current()
-  self.check_annotation(w, 1, FutureWarning,
-'fnc_test_experimental_with_current',
-'experimental', [('instead', True)])
+  self.check_annotation(warning=w, warning_size=1,
+warning_type=FutureWarning,
+fnc_name='fnc_test_experimental_with_current',
+annotation_type='experimental',
+label_check_list=[('instead', True)])
 
   def test_experimental_without_current(self):
 with warnings.catch_warnings(record=True) as w:
@@ -71,9 +78,11 @@ class AnnotationTests(unittest.TestCase):
   def fnc_test_experimental_without_current():
 return 'lol'
   fnc_test_experimental_without_current()
-  self.check_annotation(w, 1, FutureWarning,
-'fnc_test_experimental_without_current',
-'experimental', [('instead', False)])
+  self.check_annotation(warning=w, warning_size=1,
+warning_type=FutureWarning,
+fnc_name='fnc_test_experimental_without_current',
+annotation_type='experimental',
+label_check_list=[('instead', False)])
 
   def test_frequency(self):
 """Tests that the filter 'once' is sufficient to print once per
@@ -89,12 +98,16 @@ class AnnotationTests(unittest.TestCase):
   fnc_test_annotate_frequency()
   fnc_test_annotate_frequency()
   fnc2_test_annotate_frequency()
-  self.check_annotation([w[0]], 1, FutureWarning,
-

[2/3] incubator-beam git commit: Add annotation to mark deprecated or experimental APIs via decorators.

2016-09-30 Thread robertwb

Add annotation to mark deprecated or experimental APIs via decorators.


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/5e87980b
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/5e87980b
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/5e87980b

Branch: refs/heads/python-sdk
Commit: 5e87980b3cd19bd62aec585808f3f7a349486a79
Parents: dc92438
Author: Maria Garcia Herrero 
Authored: Sat Sep 24 02:05:46 2016 -0700
Committer: Robert Bradshaw 
Committed: Fri Sep 30 16:48:05 2016 -0700

--
 sdks/python/apache_beam/utils/annotations.py|  99 
 .../apache_beam/utils/annotations_test.py   | 113 +++
 2 files changed, 212 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/incubator-beam/blob/5e87980b/sdks/python/apache_beam/utils/annotations.py
--
diff --git a/sdks/python/apache_beam/utils/annotations.py 
b/sdks/python/apache_beam/utils/annotations.py
new file mode 100644
index 000..aa53554
--- /dev/null
+++ b/sdks/python/apache_beam/utils/annotations.py
@@ -0,0 +1,99 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+""" Deprecated and experimental annotations.
+
+Annotations come in two flavors: deprecated and experimental
+
+The 'deprecated' annotation requires a 'since" parameter to specify
+what version deprecated it.
+Both 'deprecated' and 'experimental' annotations can specify the
+current recommended version to use by means of a 'current' parameter.
+
+The following example illustrates how to annotate coexisting versions of the
+same function 'multiply'.
+def multiply(arg1, arg2):
+  print arg1, '*', arg2, '=',
+  return arg1*arg2
+
+# This annotation marks 'old_multiply' as deprecated since 'v.1' and suggests
+# using 'multiply' instead.
+@deprecated(since='v.1', current='multiply')
+def old_multiply(arg1, arg2):
+  result = 0
+  for i in xrange(arg1):
+  result += arg2
+  print arg1, '*', arg2, '(the old way)=',
+  return result
+
+# This annotation marks 'exp_multiply' as experimental and suggests
+# using 'multiply' instead.
+@experimental(since='v.1', current='multiply')
+def exp_multiply(arg1, arg2):
+  print arg1, '*', arg2, '(the experimental way)=',
+  return (arg1*arg2)*(arg1/arg2)*(arg2/arg1)
+
+# Set a warning filter to control how often warnings are produced
+warnings.simplefilter("always")
+print multiply(5, 6)
+print old_multiply(5,6)
+print exp_multiply(5,6)
+"""
+
+import warnings
+from functools import partial
+from functools import wraps
+
+# Produce only the first occurrence of matching warnings regardless of
+# location per line of execution. Since the number of lines of execution
+# depends on the concrete runner, the number of warnings produced will
+# vary depending on the runner.
+warnings.simplefilter("once")
+
+
+def annotate(label, since, current):
+  """Decorates a function with a deprecated or experimental annotation.
+
+  Args:
+label: the kind of annotation ('deprecated' or 'experimental').
+since: the version that causes the annotation.
+current: the suggested replacement function.
+
+  Returns:
+The decorator for the function.
+  """
+  def _annotate(fnc):
+@wraps(fnc)
+def inner(*args, **kwargs):
+  if label == 'deprecated':
+warning_type = DeprecationWarning
+  else:
+warning_type = FutureWarning
+  message = '%s is %s' % (fnc.__name__, label)
+  if label == 'deprecated':
+message += ' since %s' % since
+  message += '. Use %s instead.'% current if current else '.'
+  warnings.warn(message, warning_type)
+  return fnc(*args, **kwargs)
+return inner
+  return _annotate
+
+# Use partial application to customize each annotation.
+# 'current' will be optional in both deprecated and experimental
+# while 'since' will be mandatory for deprecated.
+deprecated = partial(annotate, label='deprecated', current=None)
+experimental = partial(annotate, label='experimental', current=None, 
since=Non

[3/3] incubator-beam git commit: Closes #1001

2016-09-30 Thread robertwb

Closes #1001


Project: http://git-wip-us.apache.org/repos/asf/incubator-beam/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-beam/commit/731a7715
Tree: http://git-wip-us.apache.org/repos/asf/incubator-beam/tree/731a7715
Diff: http://git-wip-us.apache.org/repos/asf/incubator-beam/diff/731a7715

Branch: refs/heads/python-sdk
Commit: 731a7715233f91599dd10133c58cdf97c87d6564
Parents: dc92438 f94eb53
Author: Robert Bradshaw 
Authored: Fri Sep 30 16:48:15 2016 -0700
Committer: Robert Bradshaw 
Committed: Fri Sep 30 16:48:15 2016 -0700

--
 sdks/python/apache_beam/utils/annotations.py|  99 +++
 .../apache_beam/utils/annotations_test.py   | 126 +++
 2 files changed, 225 insertions(+)
--

Jenkins build is back to normal : beam_PostCommit_MavenVerify » Apache Beam :: Parent #1431

2016-09-30 Thread Apache Jenkins Server

See

Jenkins build is back to normal : beam_PostCommit_MavenVerify #1431

2016-09-30 Thread Apache Jenkins Server

See

Jenkins build is back to stable : beam_PostCommit_RunnableOnService_GoogleCloudDataflow #1245

2016-09-30 Thread Apache Jenkins Server

See

37 matches

Mail list logo