Build failed in Jenkins: beam_PostRelease_NightlySnapshot #104

[Apache Jenkins Server]

See 


--
[...truncated 722.07 KB...]

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (3s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (4s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (5s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (6s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (7s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (8s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (9s) Current status: 
PENDING

  
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (10s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (11s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (12s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (13s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (14s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (15s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (16s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (17s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (18s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (19s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (20s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (21s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (22s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (23s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (24s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (25s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (26s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (27s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (28s) Current status: 
PENDING

   
Waiting on bqjob_r7b172682e8e951bc_0161ff685458_1 ... (29s) Current status: 
PENDING
 

[beam] branch release-2.4.0 updated: [maven-release-plugin] rollback changes from release preparation of v2.4.0-RC1

[robertwb]

This is an automated email from the ASF dual-hosted git repository.

robertwb pushed a commit to branch release-2.4.0
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/release-2.4.0 by this push:
 new 9754350  [maven-release-plugin] rollback changes from release 
preparation of v2.4.0-RC1
9754350 is described below

commit 9754350e5955b56359e96e61350ef24b36b7f4f9
Author: Robert Bradshaw 
AuthorDate: Tue Mar 6 23:03:39 2018 -0800

[maven-release-plugin] rollback changes from release preparation of 
v2.4.0-RC1
---
 examples/java/pom.xml   | 2 +-
 examples/pom.xml| 2 +-
 model/fn-execution/pom.xml  | 2 +-
 model/job-management/pom.xml| 2 +-
 model/pipeline/pom.xml  | 2 +-
 model/pom.xml   | 2 +-
 pom.xml | 4 ++--
 runners/apex/pom.xml| 2 +-
 runners/core-construction-java/pom.xml  | 2 +-
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/gcp/gcemd/pom.xml   | 2 +-
 runners/gcp/gcsproxy/pom.xml| 2 +-
 runners/gcp/pom.xml | 2 +-
 runners/gearpump/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/java-fn-execution/pom.xml   | 2 +-
 runners/local-artifact-service-java/pom.xml | 2 +-
 runners/local-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/reference/java/pom.xml  | 2 +-
 runners/reference/job-server/pom.xml| 2 +-
 runners/reference/pom.xml   | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/go/pom.xml | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/container/pom.xml | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/google-cloud-platform-core/pom.xml | 2 +-
 sdks/java/extensions/jackson/pom.xml| 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/extensions/protobuf/pom.xml   | 2 +-
 sdks/java/extensions/sketching/pom.xml  | 2 +-
 sdks/java/extensions/sorter/pom.xml | 2 +-
 sdks/java/extensions/sql/pom.xml| 2 +-
 sdks/java/fn-execution/pom.xml  | 2 +-
 sdks/java/harness/pom.xml   | 2 +-
 sdks/java/io/amazon-web-services/pom.xml| 2 +-
 sdks/java/io/amqp/pom.xml   | 2 +-
 sdks/java/io/cassandra/pom.xml  | 2 +-
 sdks/java/io/common/pom.xml | 2 +-
 sdks/java/io/elasticsearch-tests/elasticsearch-tests-2/pom.xml  | 2 +-
 sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/pom.xml  | 2 +-
 sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/pom.xml | 2 +-
 sdks/java/io/elasticsearch-tests/pom.xml| 2 +-
 sdks/java/io/elasticsearch/pom.xml  | 2 +-
 sdks/java/io/file-based-io-tests/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hadoop-common/pom.xml  | 2 +-
 sdks/java/io/hadoop-file-system/pom.xml | 2 +-
 sdks/java/io/hadoop-input-format/pom.xml| 2 +-
 sdks/java/io/hbase/pom.xml  | 2 +-
 sdks/java/io/hcatalog/pom.xml   | 2 +-
 sdks/java/io/jdbc/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml   

[beam] annotated tag v2.4.0-RC1 created (now 112cbc9)

[robertwb]

This is an automated email from the ASF dual-hosted git repository.

robertwb pushed a change to annotated tag v2.4.0-RC1
in repository https://gitbox.apache.org/repos/asf/beam.git.


  at 112cbc9  (tag)
 tagging 8336466825b6340b04f74ed047ab8a10ec30e911 (commit)
 replaces OldDoFn
  by Robert Bradshaw
  on Tue Mar 6 22:31:19 2018 -0800

- Log -
[maven-release-plugin] copy for tag v2.4.0-RC1
---

No new revisions were added by this update.

-- 
To stop receiving notification emails like this one, please contact
rober...@apache.org.

[beam-site] 01/02: Update site to mention the 2 Python lint envs

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 158e256eb11338c1324694892273e2bd38a192fc
Author: Holden Karau 
AuthorDate: Thu Mar 1 18:37:10 2018 -0800

Update site to mention the 2 Python lint envs
---
 content/contribute/contribution-guide/index.html | 2 +-
 src/contribute/contribution-guide.md | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/content/contribute/contribution-guide/index.html 
b/content/contribute/contribution-guide/index.html
index ec6addf..fa8ebe8 100644
--- a/content/contribute/contribution-guide/index.html
+++ b/content/contribute/contribution-guide/index.html
@@ -435,7 +435,7 @@ environment before testing your code.
 command.
 
 $ pip install tox
-$ tox -e lint
+$ tox -e lint_py2,lint_py3
 
 
 
diff --git a/src/contribute/contribution-guide.md 
b/src/contribute/contribution-guide.md
index 5a7f0b9..fcf7502 100644
--- a/src/contribute/contribution-guide.md
+++ b/src/contribute/contribution-guide.md
@@ -248,7 +248,8 @@ To Check for lint errors locally, install "tox" package and 
run following
 command.
 
 $ pip install tox
-$ tox -e lint
+$ tox -e lint_py2,lint_py3
+
 
 Beam supports running Python SDK tests using Maven. For this, navigate to root
 directory of your Apache Beam clone and execute following command. Currently

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[beam-site] branch mergebot updated (450b3ca -> 45345f8)

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a change to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git.


 discard 450b3ca  This closes #396
 discard f92424f  Update site to mention the 2 Python lint envs
 new 158e256  Update site to mention the 2 Python lint envs
 new 45345f8  This closes #396

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (450b3ca)
\
 N -- N -- N   refs/heads/mergebot (45345f8)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[beam-site] 02/02: This closes #396

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 45345f89a04ae1a5c9e1316eb3ebd5c0bac54014
Merge: d4609c5 158e256
Author: Mergebot 
AuthorDate: Tue Mar 6 22:54:17 2018 -0800

This closes #396

 content/contribute/contribution-guide/index.html | 2 +-
 src/contribute/contribution-guide.md | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

Jenkins build is back to normal : beam_PostRelease_NightlySnapshot #103

[Apache Jenkins Server]

See 

[beam] branch release-2.4.0 updated (12fe363 -> 8336466)

[robertwb]

This is an automated email from the ASF dual-hosted git repository.

robertwb pushed a change to branch release-2.4.0
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 12fe363  [maven-release-plugin] prepare branch release-2.4.0
 new 1b2ba1e  Revert "extracting the scheduled executor service in a 
factory variable in SDF direct runner factory"
 new 8336466  [maven-release-plugin] prepare release v2.4.0-RC1

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 examples/java/pom.xml  |   2 +-
 examples/pom.xml   |   2 +-
 model/fn-execution/pom.xml |   2 +-
 model/job-management/pom.xml   |   2 +-
 model/pipeline/pom.xml |   2 +-
 model/pom.xml  |   2 +-
 pom.xml|   4 +-
 runners/apex/pom.xml   |   2 +-
 runners/core-construction-java/pom.xml |   2 +-
 runners/core-java/pom.xml  |   2 +-
 .../org/apache/beam/runners/core/DoFnRunners.java  |   2 +-
 .../apache/beam/runners/core/ProcessFnRunner.java  |  10 +-
 .../apache/beam/runners/core/SimpleDoFnRunner.java |   4 -
 runners/direct-java/pom.xml|   2 +-
 ...LifecycleManagerRemovingTransformEvaluator.java |   4 -
 .../direct/ExecutorServiceParallelExecutor.java|  40 ++--
 .../apache/beam/runners/direct/ParDoEvaluator.java |   8 --
 .../beam/runners/direct/ParDoEvaluatorFactory.java |   2 +-
 .../SplittableProcessElementsEvaluatorFactory.java | 109 +++--
 .../beam/runners/direct/DirectRunnerTest.java  |  35 ---
 runners/flink/pom.xml  |   2 +-
 runners/gcp/gcemd/pom.xml  |   2 +-
 runners/gcp/gcsproxy/pom.xml   |   2 +-
 runners/gcp/pom.xml|   2 +-
 runners/gearpump/pom.xml   |   2 +-
 runners/google-cloud-dataflow-java/pom.xml |   2 +-
 runners/java-fn-execution/pom.xml  |   2 +-
 runners/local-artifact-service-java/pom.xml|   2 +-
 runners/local-java/pom.xml |   2 +-
 runners/pom.xml|   2 +-
 runners/reference/java/pom.xml |   2 +-
 runners/reference/job-server/pom.xml   |   2 +-
 runners/reference/pom.xml  |   2 +-
 runners/spark/pom.xml  |   2 +-
 sdks/go/pom.xml|   2 +-
 sdks/java/build-tools/pom.xml  |   2 +-
 sdks/java/container/pom.xml|   2 +-
 sdks/java/core/pom.xml |   2 +-
 .../extensions/google-cloud-platform-core/pom.xml  |   2 +-
 sdks/java/extensions/jackson/pom.xml   |   2 +-
 sdks/java/extensions/join-library/pom.xml  |   2 +-
 sdks/java/extensions/pom.xml   |   2 +-
 sdks/java/extensions/protobuf/pom.xml  |   2 +-
 sdks/java/extensions/sketching/pom.xml |   2 +-
 sdks/java/extensions/sorter/pom.xml|   2 +-
 sdks/java/extensions/sql/pom.xml   |   2 +-
 sdks/java/fn-execution/pom.xml |   2 +-
 sdks/java/harness/pom.xml  |   2 +-
 sdks/java/io/amazon-web-services/pom.xml   |   2 +-
 sdks/java/io/amqp/pom.xml  |   2 +-
 sdks/java/io/cassandra/pom.xml |   2 +-
 sdks/java/io/common/pom.xml|   2 +-
 .../elasticsearch-tests-2/pom.xml  |   2 +-
 .../elasticsearch-tests-5/pom.xml  |   2 +-
 .../elasticsearch-tests-common/pom.xml |   2 +-
 sdks/java/io/elasticsearch-tests/pom.xml   |   2 +-
 sdks/java/io/elasticsearch/pom.xml |   2 +-
 sdks/java/io/file-based-io-tests/pom.xml   |   2 +-
 sdks/java/io/google-cloud-platform/pom.xml |   2 +-
 sdks/java/io/hadoop-common/pom.xml |   2 +-
 sdks/java/io/hadoop-file-system/pom.xml|   2 +-
 sdks/java/io/hadoop-input-format/pom.xml   |   2 +-
 sdks/java/io/hbase/pom.xml |   2 +-
 sdks/java/io/hcatalog/pom.xml  |   2 +-
 sdks/java/io/jdbc/pom.xml  |   2 +-
 sdks/java/io/jms/pom.xml   |   2 +-
 sdks/java/io/kafka/pom.xml |   2 +-
 sdks/java/io/kinesis/pom.xml   |   2 +-
 sdks/java/io/mongodb/pom.xml   |   2 +-
 sdks/java/io/mqtt/pom.xml  |   2 +-
 

[beam] 01/02: Revert "extracting the scheduled executor service in a factory variable in SDF direct runner factory"

[robertwb]

This is an automated email from the ASF dual-hosted git repository.

robertwb pushed a commit to branch release-2.4.0
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 1b2ba1e2b1a183010ee1ce4d317592ea0386b41b
Author: Thomas Groh 
AuthorDate: Fri Mar 2 09:27:48 2018 -0800

Revert "extracting the scheduled executor service in a factory variable in 
SDF direct runner factory"

This reverts commit 0044cbf385a4991a7d5191a91b881d8525d747c0.

Breaks FlinkRunner Compilation
---
 .../org/apache/beam/runners/core/DoFnRunners.java  |   2 +-
 .../apache/beam/runners/core/ProcessFnRunner.java  |  10 +-
 .../apache/beam/runners/core/SimpleDoFnRunner.java |   4 -
 ...LifecycleManagerRemovingTransformEvaluator.java |   4 -
 .../direct/ExecutorServiceParallelExecutor.java|  40 ++--
 .../apache/beam/runners/direct/ParDoEvaluator.java |   8 --
 .../beam/runners/direct/ParDoEvaluatorFactory.java |   2 +-
 .../SplittableProcessElementsEvaluatorFactory.java | 109 +++--
 .../beam/runners/direct/DirectRunnerTest.java  |  35 ---
 9 files changed, 67 insertions(+), 147 deletions(-)

diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunners.java 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunners.java
index 41116f1..80c830a 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunners.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/DoFnRunners.java
@@ -52,7 +52,7 @@ public class DoFnRunners {
* compressed {@link WindowedValue}. It is the responsibility of the runner 
to perform any key
* partitioning needed, etc.
*/
-  public static  SimpleDoFnRunner 
simpleRunner(
+  public static  DoFnRunner simpleRunner(
   PipelineOptions options,
   DoFn fn,
   SideInputReader sideInputReader,
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
index 6690f58..e4dfd13 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/ProcessFnRunner.java
@@ -25,7 +25,6 @@ import java.util.Collections;
 import org.apache.beam.runners.core.StateNamespaces.WindowNamespace;
 import org.apache.beam.runners.core.TimerInternals.TimerData;
 import org.apache.beam.sdk.state.TimeDomain;
-import org.apache.beam.sdk.transforms.DoFn;
 import org.apache.beam.sdk.transforms.windowing.BoundedWindow;
 import org.apache.beam.sdk.transforms.windowing.GlobalWindow;
 import org.apache.beam.sdk.util.WindowedValue;
@@ -40,13 +39,12 @@ import org.joda.time.Instant;
 public class ProcessFnRunner
 implements PushbackSideInputDoFnRunner<
 KeyedWorkItem>, OutputT> {
-  private final SimpleDoFnRunner>, OutputT>
-underlying;
+  private final DoFnRunner>, 
OutputT> underlying;
   private final Collection views;
   private final ReadyCheckingSideInputReader sideInputReader;
 
   public ProcessFnRunner(
-  SimpleDoFnRunner>, 
OutputT> underlying,
+  DoFnRunner>, OutputT> 
underlying,
   Collection views,
   ReadyCheckingSideInputReader sideInputReader) {
 this.underlying = underlying;
@@ -54,10 +52,6 @@ public class ProcessFnRunner
 this.sideInputReader = sideInputReader;
   }
 
-  public DoFn>, OutputT> 
getFn() {
-return underlying.getFn();
-  }
-
   @Override
   public void startBundle() {
 underlying.startBundle();
diff --git 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
index 36b42ef..d4c5775 100644
--- 
a/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
+++ 
b/runners/core-java/src/main/java/org/apache/beam/runners/core/SimpleDoFnRunner.java
@@ -120,10 +120,6 @@ public class SimpleDoFnRunner implements 
DoFnRunner getFn() {
-return fn;
-  }
-
   @Override
   public void startBundle() {
 // This can contain user code. Wrap it in case it throws an exception.
diff --git 
a/runners/direct-java/src/main/java/org/apache/beam/runners/direct/DoFnLifecycleManagerRemovingTransformEvaluator.java
 

[beam] 02/02: [maven-release-plugin] prepare release v2.4.0-RC1

[robertwb]

This is an automated email from the ASF dual-hosted git repository.

robertwb pushed a commit to branch release-2.4.0
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 8336466825b6340b04f74ed047ab8a10ec30e911
Author: Robert Bradshaw 
AuthorDate: Tue Mar 6 22:28:54 2018 -0800

[maven-release-plugin] prepare release v2.4.0-RC1
---
 examples/java/pom.xml   | 2 +-
 examples/pom.xml| 2 +-
 model/fn-execution/pom.xml  | 2 +-
 model/job-management/pom.xml| 2 +-
 model/pipeline/pom.xml  | 2 +-
 model/pom.xml   | 2 +-
 pom.xml | 4 ++--
 runners/apex/pom.xml| 2 +-
 runners/core-construction-java/pom.xml  | 2 +-
 runners/core-java/pom.xml   | 2 +-
 runners/direct-java/pom.xml | 2 +-
 runners/flink/pom.xml   | 2 +-
 runners/gcp/gcemd/pom.xml   | 2 +-
 runners/gcp/gcsproxy/pom.xml| 2 +-
 runners/gcp/pom.xml | 2 +-
 runners/gearpump/pom.xml| 2 +-
 runners/google-cloud-dataflow-java/pom.xml  | 2 +-
 runners/java-fn-execution/pom.xml   | 2 +-
 runners/local-artifact-service-java/pom.xml | 2 +-
 runners/local-java/pom.xml  | 2 +-
 runners/pom.xml | 2 +-
 runners/reference/java/pom.xml  | 2 +-
 runners/reference/job-server/pom.xml| 2 +-
 runners/reference/pom.xml   | 2 +-
 runners/spark/pom.xml   | 2 +-
 sdks/go/pom.xml | 2 +-
 sdks/java/build-tools/pom.xml   | 2 +-
 sdks/java/container/pom.xml | 2 +-
 sdks/java/core/pom.xml  | 2 +-
 sdks/java/extensions/google-cloud-platform-core/pom.xml | 2 +-
 sdks/java/extensions/jackson/pom.xml| 2 +-
 sdks/java/extensions/join-library/pom.xml   | 2 +-
 sdks/java/extensions/pom.xml| 2 +-
 sdks/java/extensions/protobuf/pom.xml   | 2 +-
 sdks/java/extensions/sketching/pom.xml  | 2 +-
 sdks/java/extensions/sorter/pom.xml | 2 +-
 sdks/java/extensions/sql/pom.xml| 2 +-
 sdks/java/fn-execution/pom.xml  | 2 +-
 sdks/java/harness/pom.xml   | 2 +-
 sdks/java/io/amazon-web-services/pom.xml| 2 +-
 sdks/java/io/amqp/pom.xml   | 2 +-
 sdks/java/io/cassandra/pom.xml  | 2 +-
 sdks/java/io/common/pom.xml | 2 +-
 sdks/java/io/elasticsearch-tests/elasticsearch-tests-2/pom.xml  | 2 +-
 sdks/java/io/elasticsearch-tests/elasticsearch-tests-5/pom.xml  | 2 +-
 sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/pom.xml | 2 +-
 sdks/java/io/elasticsearch-tests/pom.xml| 2 +-
 sdks/java/io/elasticsearch/pom.xml  | 2 +-
 sdks/java/io/file-based-io-tests/pom.xml| 2 +-
 sdks/java/io/google-cloud-platform/pom.xml  | 2 +-
 sdks/java/io/hadoop-common/pom.xml  | 2 +-
 sdks/java/io/hadoop-file-system/pom.xml | 2 +-
 sdks/java/io/hadoop-input-format/pom.xml| 2 +-
 sdks/java/io/hbase/pom.xml  | 2 +-
 sdks/java/io/hcatalog/pom.xml   | 2 +-
 sdks/java/io/jdbc/pom.xml   | 2 +-
 sdks/java/io/jms/pom.xml| 2 +-
 sdks/java/io/kafka/pom.xml  | 2 +-
 sdks/java/io/kinesis/pom.xml| 2 +-
 sdks/java/io/mongodb/pom.xml| 2 +-
 sdks/java/io/mqtt/pom.xml  

Build failed in Jenkins: beam_PerformanceTests_Spark #1438

[Apache Jenkins Server]

See 


Changes:

[kenn] Bump sdks/go/container.pom.xml to 2.5.0-SNAPSHOT

[amyrvold] [BEAM-3791] Update version number in build_rules.gradle

--
[...truncated 95.68 KB...]
'apache-beam-testing:bqjob_r34650bc0f3860108_0161ff1e_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: 
/usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
 UserWarning: You have requested explicit scopes to be used with a GCE service 
account.
Using this argument will have no effect on the actual scopes for tokens
requested. These scopes are set at VM instance creation time and
can't be overridden in the request.

  warnings.warn(_SCOPES_WARNING)

2018-03-07 06:18:54,903 e14374c0 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-03-07 06:19:13,355 e14374c0 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-03-07 06:19:15,807 e14374c0 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1,  WallTime:0:02.44s,  CPU:0.35s,  MaxMemory:25136kb 
STDOUT: Upload complete.
Waiting on bqjob_r598cec2fa9d38ab2_0161ff1d1da2_1 ... (0s) Current status: 
RUNNING 
 Waiting on bqjob_r598cec2fa9d38ab2_0161ff1d1da2_1 ... (0s) 
Current status: DONE   
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_r598cec2fa9d38ab2_0161ff1d1da2_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: 
/usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
 UserWarning: You have requested explicit scopes to be used with a GCE service 
account.
Using this argument will have no effect on the actual scopes for tokens
requested. These scopes are set at VM instance creation time and
can't be overridden in the request.

  warnings.warn(_SCOPES_WARNING)

2018-03-07 06:19:15,807 e14374c0 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-03-07 06:19:41,885 e14374c0 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-03-07 06:19:44,423 e14374c0 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1,  WallTime:0:02.52s,  CPU:0.35s,  MaxMemory:25516kb 
STDOUT: Upload complete.
Waiting on bqjob_rb99ef234933ee2e_0161ff1d8d33_1 ... (0s) Current status: 
RUNNING 
Waiting on bqjob_rb99ef234933ee2e_0161ff1d8d33_1 ... (0s) 
Current status: DONE   
BigQuery error in load operation: Error processing job
'apache-beam-testing:bqjob_rb99ef234933ee2e_0161ff1d8d33_1': Invalid schema
update. Field timestamp has changed type from TIMESTAMP to FLOAT

STDERR: 
/usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
 UserWarning: You have requested explicit scopes to be used with a GCE service 
account.
Using this argument will have no effect on the actual scopes for tokens
requested. These scopes are set at VM instance creation time and
can't be overridden in the request.

  warnings.warn(_SCOPES_WARNING)

2018-03-07 06:19:44,424 e14374c0 MainThread INFO Retrying exception running 
IssueRetryableCommand: Command returned a non-zero exit code.

2018-03-07 06:20:03,014 e14374c0 MainThread INFO Running: bq load 
--autodetect --source_format=NEWLINE_DELIMITED_JSON 
beam_performance.pkb_results 

2018-03-07 06:20:06,387 e14374c0 MainThread INFO Ran: {bq load --autodetect 
--source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results 

  ReturnCode:1,  WallTime:0:03.26s,  CPU:0.35s,  MaxMemory:25252kb 
STDOUT: Upload complete.
Waiting on bqjob_r7d32e99b2d606840_0161ff1de340_1 ... (0s) Current status: 
RUNNING 
 Waiting on 

Jenkins build is back to normal : beam_PerformanceTests_JDBC #298

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PerformanceTests_Python #994

[Apache Jenkins Server]

See 


Changes:

[kenn] Bump sdks/go/container.pom.xml to 2.5.0-SNAPSHOT

[amyrvold] [BEAM-3791] Update version number in build_rules.gradle

--
[...truncated 1.26 KB...]
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins2264025763315451344.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins4919913635282871360.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins8706687772109012085.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins5344082896574439861.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from 
https://pypi.python.org/packages/ad/dc/fcced9ec3f2561c0cbe8eb6527eef7cf4f4919a2b3a07891a36e846635af/setuptools-38.5.2-py2.py3-none-any.whl#md5=abd3307cdce6fb543b5a4d0e3e98bdb6
Downloading/unpacking pip from 
https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
  Found existing installation: setuptools 2.2
Uninstalling setuptools:
  Successfully uninstalled setuptools
  Found existing installation: pip 1.5.4
Uninstalling pip:
  Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins2711624086536140579.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins8434680428454394653.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in 
/usr/local/lib/python2.7/dist-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages 
(from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in 
/usr/local/lib/python2.7/dist-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in 
/usr/local/lib/python2.7/dist-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages 
(from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in 
/usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in 
/usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r 
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in 
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in 
/usr/local/lib/python2.7/dist-packages (from 

Jenkins build is back to normal : beam_PerformanceTests_AvroIOIT #224

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #102

[Apache Jenkins Server]

See 


Changes:

[yifanzou] BEAM-3339 Mobile gaming automation for Java nightly snapshot

--
[...truncated 5.40 MB...]

org.apache.beam.examples.complete.game.utils.WriteToBigQuery$BuildRowFn$DoFnInvoker.invokeProcessElement(Unknown
 Source)

org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:177)

org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:138)

com.google.cloud.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:323)

com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)

com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)

com.google.cloud.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:181)

com.google.cloud.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:101)

org.apache.beam.runners.core.ReduceFnRunner.lambda$onTrigger$1(ReduceFnRunner.java:1040)

org.apache.beam.runners.core.ReduceFnContextFactory$OnTriggerContextImpl.output(ReduceFnContextFactory.java:433)

org.apache.beam.runners.core.SystemReduceFn.onTrigger(SystemReduceFn.java:127)

org.apache.beam.runners.core.ReduceFnRunner.onTrigger(ReduceFnRunner.java:1043)

org.apache.beam.runners.core.ReduceFnRunner.emit(ReduceFnRunner.java:911)

org.apache.beam.runners.core.ReduceFnRunner.processElements(ReduceFnRunner.java:362)

com.google.cloud.dataflow.worker.StreamingGroupAlsoByWindowViaWindowSetFn.processElement(StreamingGroupAlsoByWindowViaWindowSetFn.java:94)

com.google.cloud.dataflow.worker.StreamingGroupAlsoByWindowViaWindowSetFn.processElement(StreamingGroupAlsoByWindowViaWindowSetFn.java:42)

com.google.cloud.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:114)

com.google.cloud.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:72)

org.apache.beam.runners.core.LateDataDroppingDoFnRunner.processElement(LateDataDroppingDoFnRunner.java:74)

com.google.cloud.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:133)

com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:43)

com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:48)

com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:189)

com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:150)

com.google.cloud.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:74)

com.google.cloud.dataflow.worker.StreamingDataflowWorker.process(StreamingDataflowWorker.java:1208)

com.google.cloud.dataflow.worker.StreamingDataflowWorker.access$1000(StreamingDataflowWorker.java:136)

com.google.cloud.dataflow.worker.StreamingDataflowWorker$6.run(StreamingDataflowWorker.java:956)

java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: 
com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad 
Request
{
  "code" : 400,
  "errors" : [ {
"domain" : "global",
"message" : "Invalid table ID \"leaderboard_dataflow-runner_user\". Table 
IDs must be alphanumeric (plus underscores) and must be at most 1024 characters 
long. Also, Table decorators cannot be used.",
"reason" : "invalid"
  } ],
  "message" : "Invalid table ID \"leaderboard_dataflow-runner_user\". Table IDs 
must be alphanumeric (plus underscores) and must be at most 1024 characters 
long. Also, Table decorators cannot be used."
}

org.apache.beam.sdk.io.gcp.bigquery.CreateTables$CreateTablesFn.tryCreateTable(CreateTables.java:184)

org.apache.beam.sdk.io.gcp.bigquery.CreateTables$CreateTablesFn.getTableDestination(CreateTables.java:143)

org.apache.beam.sdk.io.gcp.bigquery.CreateTables$CreateTablesFn.lambda$processElement$0(CreateTables.java:106)
java.util.HashMap.computeIfAbsent(HashMap.java:1126)

org.apache.beam.sdk.io.gcp.bigquery.CreateTables$CreateTablesFn.processElement(CreateTables.java:105)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 
400 Bad Request
{
  "code" : 400,
  "errors" : [ {
"domain" : "global",
"message" : "Invalid table ID \"leaderboard_dataflow-runner_user\". Table 
IDs must be alphanumeric (plus 

Jenkins build is back to normal : beam_PostRelease_NightlySnapshot #101

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostCommit_Python_Verify #4379

[Apache Jenkins Server]

See 


--
[...truncated 1.02 MB...]
copying 
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers_test.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying apache_beam/runners/dataflow/native_io/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/streaming_create.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/bundle_factory.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/clock.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor_test.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/evaluation_context.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/executor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/helper_transforms.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/transform_evaluator.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying 
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying 

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1063

[Apache Jenkins Server]

See 


--
[...truncated 117.17 KB...]
self._actions = []
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 
"
 line 812, in run
test(orig)
  File 
"
 line 45, in __call__
return self.run(*arg, **kwarg)
  File 
"
 line 133, in run
self.runTest(result)
  File 
"
 line 151, in runTest
test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
  File 
"
 line 178, in test_iterable_side_input
pipeline.run()
  File 
"
 line 102, in run
result = super(TestPipeline, self).run()
  File 
"
 line 367, in run
if test_runner_api and self._verify_runner_api_compatible():
  File 
"
 line 570, in _verify_runner_api_compatible
self.visit(Visitor())
  File 
"
 line 410, in visit
self._root_transform().visit(visitor, self, visited)
  File 
"
 line 764, in visit
part.visit(visitor, pipeline, visited)
  File 
"
 line 764, in visit
part.visit(visitor, pipeline, visited)
  File 
"
 line 767, in visit
visitor.visit_transform(self)
  File 
"
 line 562, in visit_transform
enable_trace=False)
  File 
"
 line 221, in loads
return dill.loads(s)
  File 
"
 line 277, in loads
return load(file)
  File 
"
 line 266, in load
obj = pik.load()
  File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
  File "/usr/lib/python2.7/pickle.py", line 1255, in load_stop
value = self.stack.pop()
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_multi_valued_singleton_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 

Build failed in Jenkins: beam_PostCommit_Python_Verify #4378

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3791] Update version number in build_rules.gradle

--
[...truncated 1.02 MB...]
copying apache_beam/runners/direct/transform_evaluator.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying 
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operation_specs.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_fast.pyx -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_slow.py -> 

[jira] [Resolved] (BEAM-3790) DirectRunner does not distinguish metrics based on namespace

[Kenneth Knowles (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kenneth Knowles resolved BEAM-3790.
---
   Resolution: Cannot Reproduce
Fix Version/s: Not applicable

Wrote some tests to confirm - doesn't repro.

> DirectRunner does not distinguish metrics based on namespace
> 
>
> Key: BEAM-3790
> URL: https://issues.apache.org/jira/browse/BEAM-3790
> Project: Beam
>  Issue Type: Bug
>  Components: runner-direct
>Reporter: Kenneth Knowles
>Assignee: Kenneth Knowles
>Priority: Critical
> Fix For: Not applicable
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : beam_PostRelease_NightlySnapshot #99

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #98

[Apache Jenkins Server]

See 


--
[...truncated 158.36 KB...]

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (44s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (45s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (46s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (47s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (48s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (49s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (50s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (51s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (52s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (53s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (54s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (55s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (56s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (57s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (58s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (59s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (60s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (61s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (62s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (63s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (64s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (65s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (66s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (67s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (68s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (69s) Current status: 
PENDING

   
Waiting on bqjob_r70cb83a53faf2365_0161fe3be64a_1 ... (70s) Current status: 
PENDING
   

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1062

[Apache Jenkins Server]

See 


Changes:

[kenn] Bump sdks/go/container.pom.xml to 2.5.0-SNAPSHOT

[amyrvold] [BEAM-3791] Update version number in build_rules.gradle

--
[...truncated 115.32 KB...]
result = p.apply(self, pvalueish, label)
  File 
"
 line 443, in apply
return self.apply(transform, pvalueish)
  File 
"
 line 479, in apply
pvalueish_result = self.runner.apply(transform, pvalueish)
  File 
"
 line 174, in apply
return m(transform, input)
  File 
"
 line 180, in apply_PTransform
return transform.expand(input)
  File 
"
 line 165, in expand
| Map(_merge_tagged_vals_under_key, result_ctor, result_ctor_arg))
  File 
"
 line 107, in __or__
return self.pipeline.apply(ptransform, self)
  File 
"
 line 433, in apply
label or transform.label)
  File 
"
 line 443, in apply
return self.apply(transform, pvalueish)
  File 
"
 line 493, in apply
self._infer_result_type(transform, inputs, result)
  File 
"
 line 535, in _infer_result_type
input_element_type)
  File 
"
 line 798, in infer_output_type
self.fn.infer_output_type(input_type))
  File 
"
 line 453, in infer_output_type
trivial_inference.infer_return_type(self._fn, [input_type]))
  File 
"
 line 240, in infer_return_type
return infer_return_type_func(c, input_types, debug, depth)
  File 
"
 line 354, in infer_return_type_func
state |= states[start]
  File 
"
 line 144, in __or__
def __or__(self, other):
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 
"
 line 812, in run
test(orig)
  File 
"
 line 45, in __call__
return self.run(*arg, **kwarg)
  File 
"
 line 133, in run
self.runTest(result)
  File 
"
 line 151, in runTest
test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return 

Jenkins build is back to normal : beam_PostCommit_Java_MavenInstall #6146

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostCommit_Python_Verify #4377

[Apache Jenkins Server]

See 


Changes:

[kenn] Bump sdks/go/container.pom.xml to 2.5.0-SNAPSHOT

--
[...truncated 953.87 KB...]
test_good_main_input 
(apache_beam.typehints.typed_pipeline_test.NativeTypesTest) ... ok
test_basic_side_input_hint 
(apache_beam.typehints.typed_pipeline_test.SideInputTest) ... ok
test_default_typed_hint 
(apache_beam.typehints.typed_pipeline_test.SideInputTest) ... ok
test_default_untyped_hint 
(apache_beam.typehints.typed_pipeline_test.SideInputTest) ... ok
test_deferred_side_input_iterable 
(apache_beam.typehints.typed_pipeline_test.SideInputTest) ... ok
test_deferred_side_inputs 
(apache_beam.typehints.typed_pipeline_test.SideInputTest) ... ok
test_keyword_side_input_hint 
(apache_beam.typehints.typed_pipeline_test.SideInputTest) ... ok
test_any_compatibility 
(apache_beam.typehints.typehints_test.AnyTypeConstraintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.AnyTypeConstraintTestCase) ... 
ok
test_type_check 
(apache_beam.typehints.typehints_test.AnyTypeConstraintTestCase) ... ok
test_composite_takes_and_returns_hints 
(apache_beam.typehints.typehints_test.CombinedReturnsAndTakesTestCase) ... ok
test_enable_and_disable_type_checking_returns 
(apache_beam.typehints.typehints_test.CombinedReturnsAndTakesTestCase) ... ok
test_enable_and_disable_type_checking_takes 
(apache_beam.typehints.typehints_test.CombinedReturnsAndTakesTestCase) ... ok
test_simple_takes_and_returns_hints 
(apache_beam.typehints.typehints_test.CombinedReturnsAndTakesTestCase) ... ok
test_valid_mix_pos_and_keyword_with_both_orders 
(apache_beam.typehints.typehints_test.CombinedReturnsAndTakesTestCase) ... ok
test_getcallargs_forhints 
(apache_beam.typehints.typehints_test.DecoratorHelpers) ... ok
test_hint_helper (apache_beam.typehints.typehints_test.DecoratorHelpers) ... ok
test_positional_arg_hints 
(apache_beam.typehints.typehints_test.DecoratorHelpers) ... ok
test_compatibility (apache_beam.typehints.typehints_test.DictHintTestCase) ... 
ok
test_getitem_param_must_be_tuple 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_getitem_param_must_have_length_2 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_key_type_must_be_valid_composite_param 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_match_type_variables 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_invalid_key_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_invalid_value_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_valid_composite_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_check_valid_simple_type 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_type_checks_not_dict 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_value_type_must_be_valid_composite_param 
(apache_beam.typehints.typehints_test.DictHintTestCase) ... ok
test_compatibility (apache_beam.typehints.typehints_test.GeneratorHintTestCase) 
... ok
test_generator_argument_hint_invalid_yield_type 
(apache_beam.typehints.typehints_test.GeneratorHintTestCase) ... ok
test_generator_return_hint_invalid_yield_type 
(apache_beam.typehints.typehints_test.GeneratorHintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.GeneratorHintTestCase) ... ok
test_compatibility (apache_beam.typehints.typehints_test.IterableHintTestCase) 
... ok
test_getitem_invalid_composite_type_param 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_repr (apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_tuple_compatibility 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_must_be_iterable 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_invalid_composite_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_invalid_simple_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_valid_composite_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_type_check_violation_valid_simple_type 
(apache_beam.typehints.typehints_test.IterableHintTestCase) ... ok
test_enforce_kv_type_constraint 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_getitem_param_must_be_tuple 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_getitem_param_must_have_length_2 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_getitem_proxy_to_tuple 
(apache_beam.typehints.typehints_test.KVHintTestCase) ... ok
test_enforce_list_type_constraint_invalid_composite_type 
(apache_beam.typehints.typehints_test.ListHintTestCase) ... 

[jira] [Commented] (BEAM-3793) SDK harness should log bad gRPC responses

[Henning Rohde (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388798#comment-16388798
 ] 

Henning Rohde commented on BEAM-3793:
-

Yeah. This is in the shared code used by all SDK containers.

> SDK harness should log bad gRPC responses
> -
>
> Key: BEAM-3793
> URL: https://issues.apache.org/jira/browse/BEAM-3793
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-go
>Reporter: Ben Sidhom
>Assignee: Henning Rohde
>Priority: Minor
>  Labels: portability
>
> As of now, if a null ProvisionInfo is passed to the boot code, it panics when 
> trying to convert to JSON. It should at the very least log a useful error 
> message if a bad ProvisionInfo is received.
>  
> Sample stack trace:
> panic: reflect: call of reflect.Value.Field on zero Value
> goroutine 1 [running]:
> reflect.Value.Field(0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x81f2e0)
>  /Users/sidhom/.gradle/go/binary/1.9/go/src/reflect/value.go:759 +0x12e
> github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).marshalObject(0xc42015cb10,
>  0xc42010, 0xb7a7e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc4201bbbd0, 0x410e58)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:207
>  +0x1040
> github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).Marshal(0xc42015cb10,
>  0xb70ae0, 0xc4201ac770, 0xb7a7e0, 0x0, 0x8a53e0, 0x4c3601)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:122
>  +0xc1
> github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).MarshalToString(0xc42015cb10,
>  0xb7a7e0, 0x0, 0x1, 0x15, 0x0, 0x0)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:128
>  +0x6e
> github.com/apache/beam/sdks/java/boot/vendor/github.com/apache/beam/sdks/go/pkg/beam/provision.ProtoToJSON(0x0,
>  0x12, 0xc4201bbdc0, 0x1, 0x1)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/apache/beam/sdks/go/pkg/beam/provision/provison.go:79
>  +0x51
> main.main()
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/boot.go:80
>  +0x309



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-3793) SDK harness should log bad gRPC responses

[Henning Rohde (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henning Rohde updated BEAM-3793:

Labels: portability  (was: )

> SDK harness should log bad gRPC responses
> -
>
> Key: BEAM-3793
> URL: https://issues.apache.org/jira/browse/BEAM-3793
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-go
>Reporter: Ben Sidhom
>Assignee: Henning Rohde
>Priority: Minor
>  Labels: portability
>
> As of now, if a null ProvisionInfo is passed to the boot code, it panics when 
> trying to convert to JSON. It should at the very least log a useful error 
> message if a bad ProvisionInfo is received.
>  
> Sample stack trace:
> panic: reflect: call of reflect.Value.Field on zero Value
> goroutine 1 [running]:
> reflect.Value.Field(0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x81f2e0)
>  /Users/sidhom/.gradle/go/binary/1.9/go/src/reflect/value.go:759 +0x12e
> github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).marshalObject(0xc42015cb10,
>  0xc42010, 0xb7a7e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc4201bbbd0, 0x410e58)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:207
>  +0x1040
> github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).Marshal(0xc42015cb10,
>  0xb70ae0, 0xc4201ac770, 0xb7a7e0, 0x0, 0x8a53e0, 0x4c3601)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:122
>  +0xc1
> github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).MarshalToString(0xc42015cb10,
>  0xb7a7e0, 0x0, 0x1, 0x15, 0x0, 0x0)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:128
>  +0x6e
> github.com/apache/beam/sdks/java/boot/vendor/github.com/apache/beam/sdks/go/pkg/beam/provision.ProtoToJSON(0x0,
>  0x12, 0xc4201bbdc0, 0x1, 0x1)
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/apache/beam/sdks/go/pkg/beam/provision/provison.go:79
>  +0x51
> main.main()
>  
> /Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/boot.go:80
>  +0x309



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3793) SDK harness should log bad gRPC responses

[Ben Sidhom (JIRA)]

Ben Sidhom created BEAM-3793:


 Summary: SDK harness should log bad gRPC responses
 Key: BEAM-3793
 URL: https://issues.apache.org/jira/browse/BEAM-3793
 Project: Beam
  Issue Type: Bug
  Components: sdk-go
Reporter: Ben Sidhom
Assignee: Henning Rohde


As of now, if a null ProvisionInfo is passed to the boot code, it panics when 
trying to convert to JSON. It should at the very least log a useful error 
message if a bad ProvisionInfo is received.

 

Sample stack trace:

panic: reflect: call of reflect.Value.Field on zero Value

goroutine 1 [running]:
reflect.Value.Field(0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x81f2e0)
 /Users/sidhom/.gradle/go/binary/1.9/go/src/reflect/value.go:759 +0x12e
github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).marshalObject(0xc42015cb10,
 0xc42010, 0xb7a7e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc4201bbbd0, 0x410e58)
 
/Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:207
 +0x1040
github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).Marshal(0xc42015cb10,
 0xb70ae0, 0xc4201ac770, 0xb7a7e0, 0x0, 0x8a53e0, 0x4c3601)
 
/Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:122
 +0xc1
github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb.(*Marshaler).MarshalToString(0xc42015cb10,
 0xb7a7e0, 0x0, 0x1, 0x15, 0x0, 0x0)
 
/Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/golang/protobuf/jsonpb/jsonpb.go:128
 +0x6e
github.com/apache/beam/sdks/java/boot/vendor/github.com/apache/beam/sdks/go/pkg/beam/provision.ProtoToJSON(0x0,
 0x12, 0xc4201bbdc0, 0x1, 0x1)
 
/Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/vendor/github.com/apache/beam/sdks/go/pkg/beam/provision/provison.go:79
 +0x51
main.main()
 
/Users/sidhom/Downloads/git/beam/sdks/java/container/.gogradle/project_gopath/src/github.com/apache/beam/sdks/java/boot/boot.go:80
 +0x309



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3792) Python submits portable pipelines to the Flink-served endpoint.

[Robert Bradshaw (JIRA)]

Robert Bradshaw created BEAM-3792:
-

 Summary: Python submits portable pipelines to the Flink-served 
endpoint.
 Key: BEAM-3792
 URL: https://issues.apache.org/jira/browse/BEAM-3792
 Project: Beam
  Issue Type: Sub-task
  Components: runner-flink
Reporter: Robert Bradshaw
Assignee: Robert Bradshaw






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : beam_PostCommit_Java_ValidatesRunner_Apex #3637

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #97

[Apache Jenkins Server]

See 


--
[...truncated 208.21 KB...]
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (4s) Current status: 
PENDING

  
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (5s) Current status: 
PENDING

  
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (6s) Current status: 
PENDING

  
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (7s) Current status: 
PENDING

  
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (8s) Current status: 
PENDING

  
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (9s) Current status: 
PENDING

  
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (10s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (11s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (12s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (13s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (14s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (15s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (16s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (17s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (18s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (19s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (20s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (21s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (22s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (23s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (24s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (25s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (26s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (27s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (28s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (29s) Current status: 
PENDING

   
Waiting on bqjob_r1ccc128d13789c7c_0161fde80cf4_1 ... (30s) Current status: 
PENDING

   
Waiting on 

[beam] branch master updated (d562f65 -> 5c1e49c)

[robertwb]

This is an automated email from the ASF dual-hosted git repository.

robertwb pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from d562f65  Merge pull request #4809: Bump sdks/go/container/pom.xml to 
2.5.0-SNAPSHOT
 add a9a1fa9  [BEAM-3791] Update version number in build_rules.gradle
 new 5c1e49c  Merge pull request #4810 from alanmyrvold/alan-version

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 build_rules.gradle | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
To stop receiving notification emails like this one, please contact
rober...@apache.org.

Build failed in Jenkins: beam_PerformanceTests_Spark #1437

[Apache Jenkins Server]

See 


Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with 
java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite 
names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec 
packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session recording and CPU 

[jira] [Created] (BEAM-3791) build_rules.gradle version (2.4.0-SNAPSHOT) should match pom.xml (2.5.0-SNAPSHOT)

[Alan Myrvold (JIRA)]

Alan Myrvold created BEAM-3791:
--

 Summary: build_rules.gradle version (2.4.0-SNAPSHOT) should match 
pom.xml (2.5.0-SNAPSHOT)
 Key: BEAM-3791
 URL: https://issues.apache.org/jira/browse/BEAM-3791
 Project: Beam
  Issue Type: Bug
  Components: build-system
Reporter: Alan Myrvold
Assignee: Davor Bonaci


build_rules.gradle has:

version = "2.4.0-SNAPSHOT"

 

it should be 2.5.0-SNAPSHOT



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #96

[Apache Jenkins Server]

See 


Changes:

[yifanzou] BEAM-3339 Mobile gaming automation for Java nightly snapshot

--
[...truncated 209.75 KB...]
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (30s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (31s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (32s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (33s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (34s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (35s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (36s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (37s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (38s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (39s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (40s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (41s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (42s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (43s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (44s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (45s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (46s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (47s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (48s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (49s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (50s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (51s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (52s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (53s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (54s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (55s) Current status: 
PENDING

   
Waiting on bqjob_r62e61d9b9a9fa533_0161fdcd5ac5_1 ... (56s) Current status: 
PENDING

[beam] branch master updated (27700ef -> d562f65)

[tgroh]

This is an automated email from the ASF dual-hosted git repository.

tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 27700ef  Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout
 add 303efb1  Bump sdks/go/container.pom.xml to 2.5.0-SNAPSHOT
 new d562f65  Merge pull request #4809: Bump sdks/go/container/pom.xml to 
2.5.0-SNAPSHOT

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/go/container/pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
To stop receiving notification emails like this one, please contact
tg...@apache.org.

[beam] 01/01: Merge pull request #4809: Bump sdks/go/container/pom.xml to 2.5.0-SNAPSHOT

[tgroh]

This is an automated email from the ASF dual-hosted git repository.

tgroh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit d562f6537b991c5c7bdba9f92388b49c0a11dcd3
Merge: 27700ef 303efb1
Author: Thomas Groh 
AuthorDate: Tue Mar 6 16:09:53 2018 -0800

Merge pull request #4809: Bump sdks/go/container/pom.xml to 2.5.0-SNAPSHOT

Bump sdks/go/container/pom.xml to 2.5.0-SNAPSHOT

 sdks/go/container/pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
To stop receiving notification emails like this one, please contact
tg...@apache.org.

Build failed in Jenkins: beam_PerformanceTests_Python #993

[Apache Jenkins Server]

See 


Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with 
java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite 
names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec 
packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session recording and CPU 

Build failed in Jenkins: beam_PerformanceTests_JDBC #297

[Apache Jenkins Server]

See 


Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with 
java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite 
names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec 
packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session recording and CPU 

Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #223

[Apache Jenkins Server]

See 


Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with 
java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite 
names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec 
packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session recording and CPU 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Dataflow #5102

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 27700ef4abbb710dff0fbceb51336db8142009cc # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 

 at hudson.remoting.Channel@6659890a:beam3: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam3
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Spark #4356

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 27700ef4abbb710dff0fbceb51336db8142009cc # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c1d78c1:beam6: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam6
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at 

Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #6145

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 27700ef4abbb710dff0fbceb51336db8142009cc # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 at 
hudson.remoting.Channel@6659890a:beam3: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam3
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Apex #3636

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 27700ef4abbb710dff0fbceb51336db8142009cc # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c1d78c1:beam6: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam6
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Flink #5175

[Apache Jenkins Server]

See 


--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 27700ef4abbb710dff0fbceb51336db8142009cc # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c1d78c1:beam6: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam6
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at 

Build failed in Jenkins: beam_PostCommit_Python_Verify #4376

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3775] Increase timeout in

--
[...truncated 1.02 MB...]
copying apache_beam/runners/direct/transform_evaluator.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying 
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operation_specs.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_fast.pyx -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_slow.py -> 

[jira] [Created] (BEAM-3790) DirectRunner does not distinguish metrics based on namespace

[Kenneth Knowles (JIRA)]

Kenneth Knowles created BEAM-3790:
-

 Summary: DirectRunner does not distinguish metrics based on 
namespace
 Key: BEAM-3790
 URL: https://issues.apache.org/jira/browse/BEAM-3790
 Project: Beam
  Issue Type: Bug
  Components: runner-direct
Reporter: Kenneth Knowles
Assignee: Kenneth Knowles






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #95

[Apache Jenkins Server]

See 


--
[...truncated 245.88 KB...]
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (31s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (32s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (33s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (34s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (35s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (36s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (37s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (38s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (39s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (40s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (41s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (42s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (43s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (44s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (45s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (46s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (47s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (48s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (49s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (50s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (51s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (52s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (53s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (54s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (55s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (56s) Current status: 
PENDING

   
Waiting on bqjob_r436f32f20f955437_0161fd8b6f2a_1 ... (57s) Current status: 
PENDING

   
Waiting on 

Build failed in Jenkins: beam_PostCommit_Python_Verify #4375

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
[...truncated 1.02 MB...]
copying 
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers_test.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying apache_beam/runners/dataflow/native_io/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/streaming_create.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/bundle_factory.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/clock.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor_test.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/evaluation_context.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/executor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/helper_transforms.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/transform_evaluator.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying 
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py -> 

[jira] [Created] (BEAM-3789) [SQL] Support Nested Rows

[Anton Kedin (JIRA)]

Anton Kedin created BEAM-3789:
-

 Summary: [SQL] Support Nested Rows
 Key: BEAM-3789
 URL: https://issues.apache.org/jira/browse/BEAM-3789
 Project: Beam
  Issue Type: Improvement
  Components: dsl-sql
Reporter: Anton Kedin
Assignee: Anton Kedin


Add support for SqlTypeName.ROW



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-3788) Implement a Kafka IO for Python SDK

[Chamikara Jayalath (JIRA)]

Chamikara Jayalath created BEAM-3788:


 Summary: Implement a Kafka IO for Python SDK
 Key: BEAM-3788
 URL: https://issues.apache.org/jira/browse/BEAM-3788
 Project: Beam
  Issue Type: New Feature
  Components: sdk-py-core
Reporter: Chamikara Jayalath
Assignee: Chamikara Jayalath


This will be implemented using the Splittable DoFn framework.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1061

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3775] Increase timeout in

--
[...truncated 125.21 KB...]
result = ParDo(fn, *args, **kwargs)
  File 
"
 line 781, in __init__
super(ParDo, self).__init__(fn, *args, **kwargs)
  File 
"
 line 627, in __init__
self.fn = pickler.loads(pickler.dumps(self.fn))
  File 
"
 line 193, in dumps
s = dill.dumps(o)
  File 
"
 line 259, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
  File 
"
 line 252, in dump
pik.dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
  File 
"
 line 165, in new_save_module_dict
return old_save_module_dict(pickler, obj)
  File 
"
 line 841, in save_module_dict
StockPickler.save_dict(pickler, obj)
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 681, in _batch_setitems
save(v)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 396, in save_reduce
save(cls)
  File "/usr/lib/python2.7/pickle.py", line 277, in save
x = self.memo.get(id(obj))
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 
"
 line 812, in run
test(orig)
  File 
"
 line 45, in __call__
return self.run(*arg, **kwarg)
  File 
"
 line 133, in run
self.runTest(result)
  File 
"
 line 151, in runTest
test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
  File 
"
 line 178, in test_iterable_side_input
pipeline.run()
  File 
"
 line 102, in run
result = super(TestPipeline, self).run()
  File 
"
 line 369, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"
 line 382, in run
return self.runner.run_pipeline(self)

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #94

[Apache Jenkins Server]

See 


--
[...truncated 130.28 KB...]
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] Failed command
:runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
Mar 06, 2018 10:32:35 PM org.apache.beam.sdk.io.FileBasedSource 
getEstimatedSizeBytes
INFO: Filepattern 
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/5000_gaming_data.csv
 matched 1 files with total size 386400
Mar 06, 2018 10:32:35 PM org.apache.beam.sdk.io.FileBasedSource split
INFO: Splitting filepattern 
gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/5000_gaming_data.csv
 into bundles of size 96600 took 56 ms and produced 1 files and 4 bundles
Mar 06, 2018 10:32:42 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 5b201e27-5bdf-4778-ab92-22f462c31d5b for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@341d9473 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:32:42 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer 9c42aa51-6218-478f-a949-25a08c61a479 for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@341d9473 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:32:42 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer f92f176a-4484-432d-a5e9-a547f6ea5e85 for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@341d9473 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:32:42 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer f8debb5a-12ab-40f7-88f7-5c5667341f6e for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@341d9473 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-32-33-1/5b201e27-5bdf-4778-ab92-22f462c31d5b
Mar 06, 2018 10:32:42 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer c0155890-101b-4401-921f-42f329e071a1 for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@341d9473 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-32-33-1/c0155890-101b-4401-921f-42f329e071a1
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-32-33-1/f92f176a-4484-432d-a5e9-a547f6ea5e85
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-32-33-1/f8debb5a-12ab-40f7-88f7-5c5667341f6e
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-32-33-1/9c42aa51-6218-478f-a949-25a08c61a479
Mar 06, 2018 10:32:42 PM 
org.apache.beam.sdk.io.WriteFiles$FinalizeTempFileBundles$FinalizeFn process
INFO: Finalizing 5 file results
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
createMissingEmptyShards
INFO: Finalizing for destination null num shards 5.
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-32-33-1/5b201e27-5bdf-4778-ab92-22f462c31d5b,
 shard=4, 
window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@341d9473, 
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, 
onTimeIndex=0}} to final location 
/tmp/groovy-generated-5934039287128256735-tmpdir/word-count-beam/java-userscore-result.txt
-4-of-5
Mar 06, 2018 10:32:42 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #93

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3775] Increase timeout in

[ehudm] Add missing import statements.

[yifanzou] BEAM-3339 Mobile gaming automation for Java nightly snapshot

--
[...truncated 128.16 KB...]
INFO: Opening writer 3b9f1aab-ad82-4e7b-95c5-cce8da093813 for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:22:32 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer dd6ab796-bb1d-4290-919e-bade8f1a9dfd for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:22:32 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer a7e7b19b-cfa5-488b-955e-a27026fb6c39 for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:22:32 PM 
org.apache.beam.sdk.io.WriteFiles$WriteShardsIntoTempFilesFn processElement
INFO: Opening writer c3290c25-cd5e-4eee-b36b-d43a9c44a401 for window 
org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7 pane 
PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, onTimeIndex=0} 
destination null
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/dd6ab796-bb1d-4290-919e-bade8f1a9dfd
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/a7e7b19b-cfa5-488b-955e-a27026fb6c39
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/c3290c25-cd5e-4eee-b36b-d43a9c44a401
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$Writer close
INFO: Successfully wrote temporary file 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/3b9f1aab-ad82-4e7b-95c5-cce8da093813
Mar 06, 2018 10:22:32 PM 
org.apache.beam.sdk.io.WriteFiles$FinalizeTempFileBundles$FinalizeFn process
INFO: Finalizing 4 file results
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
createMissingEmptyShards
INFO: Finalizing for destination null num shards 4.
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/dd6ab796-bb1d-4290-919e-bade8f1a9dfd,
 shard=2, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7, 
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, 
onTimeIndex=0}} to final location 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/java-userscore-result.txt
-2-of-4
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/c3290c25-cd5e-4eee-b36b-d43a9c44a401,
 shard=0, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7, 
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, 
onTimeIndex=0}} to final location 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/java-userscore-result.txt
-0-of-4
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/a7e7b19b-cfa5-488b-955e-a27026fb6c39,
 shard=1, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7, 
paneInfo=PaneInfo{isFirst=true, isLast=true, timing=ON_TIME, index=0, 
onTimeIndex=0}} to final location 
/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/java-userscore-result.txt
-1-of-4
Mar 06, 2018 10:22:32 PM org.apache.beam.sdk.io.FileBasedSink$WriteOperation 
moveToOutputFiles
INFO: Will copy temporary file 
FileResult{tempFilename=/tmp/groovy-generated-5546354579206426905-tmpdir/word-count-beam/.temp-beam-2018-03-06_22-22-22-1/3b9f1aab-ad82-4e7b-95c5-cce8da093813,
 shard=3, window=org.apache.beam.sdk.transforms.windowing.GlobalWindow@badeba7, 

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1060

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
[...truncated 124.78 KB...]
proto_utils.parse_Bytes(proto.payload, parameter_type),
  File 
"
 line 55, in parse_Bytes
msg = msg_class()
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 
"
 line 812, in run
test(orig)
  File 
"
 line 45, in __call__
return self.run(*arg, **kwarg)
  File 
"
 line 133, in run
self.runTest(result)
  File 
"
 line 151, in runTest
test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
  File 
"
 line 178, in test_iterable_side_input
pipeline.run()
  File 
"
 line 102, in run
result = super(TestPipeline, self).run()
  File 
"
 line 369, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"
 line 382, in run
return self.runner.run_pipeline(self)
  File 
"
 line 285, in run_pipeline
return_context=True)
  File 
"
 line 580, in to_runner_api
root_transform_id = context.transforms.get_id(self._root_transform())
  File 
"
 line 60, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
  File 
"
 line 810, in to_runner_api
for part in self.parts],
  File 
"
 line 60, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
  File 
"
 line 810, in to_runner_api
for part in self.parts],
  File 
"
 line 60, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
  File 
"
 line 810, in to_runner_api
for part in self.parts],
  File 
"
 line 60, in get_id
self._id_to_proto[id] = obj.to_runner_api(self._pipeline_context)
  File 

[beam-site] 01/02: Update site to mention the 2 Python lint envs

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit f92424f751f88a50577bc756b1f7df4ec5ba8f55
Author: Holden Karau 
AuthorDate: Thu Mar 1 18:37:10 2018 -0800

Update site to mention the 2 Python lint envs
---
 content/contribute/contribution-guide/index.html | 2 +-
 src/contribute/contribution-guide.md | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/content/contribute/contribution-guide/index.html 
b/content/contribute/contribution-guide/index.html
index ec6addf..fa8ebe8 100644
--- a/content/contribute/contribution-guide/index.html
+++ b/content/contribute/contribution-guide/index.html
@@ -435,7 +435,7 @@ environment before testing your code.
 command.
 
 $ pip install tox
-$ tox -e lint
+$ tox -e lint_py2,lint_py3
 
 
 
diff --git a/src/contribute/contribution-guide.md 
b/src/contribute/contribution-guide.md
index 5a7f0b9..fcf7502 100644
--- a/src/contribute/contribution-guide.md
+++ b/src/contribute/contribution-guide.md
@@ -248,7 +248,8 @@ To Check for lint errors locally, install "tox" package and 
run following
 command.
 
 $ pip install tox
-$ tox -e lint
+$ tox -e lint_py2,lint_py3
+
 
 Beam supports running Python SDK tests using Maven. For this, navigate to root
 directory of your Apache Beam clone and execute following command. Currently

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[beam-site] branch mergebot updated (fbdc177 -> 450b3ca)

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a change to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git.


 discard fbdc177  This closes #396
 discard 51cf2e6  Update site to mention the 2 Python lint envs
 new f92424f  Update site to mention the 2 Python lint envs
 new 450b3ca  This closes #396

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (fbdc177)
\
 N -- N -- N   refs/heads/mergebot (450b3ca)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[beam-site] 02/02: This closes #396

[mergebot-role]

This is an automated email from the ASF dual-hosted git repository.

mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git

commit 450b3cac12d8189c7478360cd0ecafcc04c2660c
Merge: d4609c5 f92424f
Author: Mergebot 
AuthorDate: Tue Mar 6 14:25:06 2018 -0800

This closes #396

 content/contribute/contribution-guide/index.html | 2 +-
 src/contribute/contribution-guide.md | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
mergebot-r...@apache.org.

[jira] [Resolved] (BEAM-3099) Implement HDFS FileSystem for Python SDK

[Udi Meiri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Udi Meiri resolved BEAM-3099.
-
   Resolution: Implemented
Fix Version/s: 2.5.0

> Implement HDFS FileSystem for Python SDK
> 
>
> Key: BEAM-3099
> URL: https://issues.apache.org/jira/browse/BEAM-3099
> Project: Beam
>  Issue Type: New Feature
>  Components: sdk-py-core
>Reporter: Chamikara Jayalath
>Assignee: Udi Meiri
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Currently Java SDK has HDFS support but Python SDK does not. With current 
> portability efforts other runners may soon be able to use Python SDK. Having 
> HDFS support will allow these runners to execute large scale jobs without 
> using GCS. 
> Following suggests some libraries that can be used to connect to HDFS from 
> Python.
> http://wesmckinney.com/blog/python-hdfs-interfaces/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Dataflow #5101

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3775] Increase timeout in

--
Started by GitHub push by tgroh
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 

 at hudson.remoting.Channel@4c1d78c1:beam6: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam6
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Flink #5174

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3775] Increase timeout in

--
Started by GitHub push by tgroh
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@a7114c0:beam4: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam4
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Apex #3635

[Apache Jenkins Server]

See 


Changes:

[amyrvold] [BEAM-3775] Increase timeout in

--
Started by GitHub push by tgroh
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 27700ef4abbb710dff0fbceb51336db8142009cc (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 27700ef4abbb710dff0fbceb51336db8142009cc
Commit message: "Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout"
 > git rev-list --no-walk 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c2365d7:beam8: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam8
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

[beam] branch master updated (5cf72a4 -> 27700ef)

[tgroh]

This is an automated email from the ASF dual-hosted git repository.

tgroh pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 5cf72a4  Merge pull request #4806 from udim/unittest-main
 add 61b97c9  [BEAM-3775] Increase timeout in 
beam_PostCommit_Java_ValidatesRunner_Dataflow from 2 to 3 hours
 new 27700ef  Merge pull request #4791: Increase Dataflow ValidatesRunner 
Timeout

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../jenkins/job_beam_PostCommit_Java_ValidatesRunner_Dataflow.groovy  | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
tg...@apache.org.

[beam] 01/01: Merge pull request #4791: Increase Dataflow ValidatesRunner Timeout

[tgroh]

This is an automated email from the ASF dual-hosted git repository.

tgroh pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 27700ef4abbb710dff0fbceb51336db8142009cc
Merge: 5cf72a4 61b97c9
Author: Thomas Groh 
AuthorDate: Tue Mar 6 14:11:50 2018 -0800

Merge pull request #4791: Increase Dataflow ValidatesRunner Timeout

[BEAM-3775] Increase timeout in beam_PostCommit_Java_ValidatesRunner_…

 .../jenkins/job_beam_PostCommit_Java_ValidatesRunner_Dataflow.groovy  | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
tg...@apache.org.

Build failed in Jenkins: beam_PostCommit_Python_Verify #4374

[Apache Jenkins Server]

See 


--
[...truncated 1.02 MB...]
copying 
apache_beam/runners/dataflow/internal/clients/dataflow/message_matchers_test.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/internal/clients/dataflow
copying apache_beam/runners/dataflow/native_io/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/iobase_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/dataflow/native_io/streaming_create.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/dataflow/native_io
copying apache_beam/runners/direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/bundle_factory.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/clock.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/consumer_tracking_pipeline_visitor_test.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_metrics_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/direct_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/evaluation_context.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/executor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/helper_transforms.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/sdf_direct_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/transform_evaluator.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/util.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/direct/watermark_manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/direct
copying apache_beam/runners/experimental/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental
copying apache_beam/runners/experimental/python_rpc_direct/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying 
apache_beam/runners/experimental/python_rpc_direct/python_rpc_direct_runner.py 
-> apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/experimental/python_rpc_direct/server.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/experimental/python_rpc_direct
copying apache_beam/runners/job/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/manager.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/job/utils.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/job
copying apache_beam/runners/portability/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/fn_api_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/maptask_executor_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/portability/universal_local_runner_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/portability
copying apache_beam/runners/test/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/test
copying apache_beam/runners/worker/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying 

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #92

[Apache Jenkins Server]

See 


Changes:

[yifanzou] BEAM-3339 Mobile gaming automation for Java nightly snapshot

--
[...truncated 146.35 KB...]
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (3s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (4s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (5s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (6s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (7s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (8s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (9s) Current status: 
PENDING


Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (10s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (11s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (12s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (13s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (14s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (15s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (16s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (17s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (18s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (19s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (20s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (21s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (22s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (23s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (24s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (25s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (26s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (27s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (28s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... (29s) Current status: 
PENDING

 
Waiting on bqjob_rdc74460a8e44b1_0161fd5aab68_1 ... 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Dataflow #5100

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
Started by GitHub push by aaltay
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5cf72a4aee13c23c454ea0abe5a952b1f97458b0
Commit message: "Merge pull request #4806 from udim/unittest-main"
 > git rev-list --no-walk ddd9748fe950e0b274845567269c440ea7c7edeb # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 

 at hudson.remoting.Channel@7574707c:beam2: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam2
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Apex #3634

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
Started by GitHub push by aaltay
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5cf72a4aee13c23c454ea0abe5a952b1f97458b0
Commit message: "Merge pull request #4806 from udim/unittest-main"
 > git rev-list --no-walk ddd9748fe950e0b274845567269c440ea7c7edeb # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c2365d7:beam8: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam8
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Flink #5173

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
Started by GitHub push by aaltay
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5cf72a4aee13c23c454ea0abe5a952b1f97458b0
Commit message: "Merge pull request #4806 from udim/unittest-main"
 > git rev-list --no-walk ddd9748fe950e0b274845567269c440ea7c7edeb # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@a7114c0:beam4: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam4
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #6143

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
Started by GitHub push by aaltay
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5cf72a4aee13c23c454ea0abe5a952b1f97458b0
Commit message: "Merge pull request #4806 from udim/unittest-main"
 > git rev-list --no-walk ddd9748fe950e0b274845567269c440ea7c7edeb # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 at 
hudson.remoting.Channel@7574707c:beam2: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam2
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Spark #4354

[Apache Jenkins Server]

See 


Changes:

[ehudm] Add missing import statements.

--
Started by GitHub push by aaltay
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 5cf72a4aee13c23c454ea0abe5a952b1f97458b0 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5cf72a4aee13c23c454ea0abe5a952b1f97458b0
Commit message: "Merge pull request #4806 from udim/unittest-main"
 > git rev-list --no-walk ddd9748fe950e0b274845567269c440ea7c7edeb # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c1d78c1:beam6: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam6
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

[beam] branch master updated (ddd9748 -> 5cf72a4)

[altay]

This is an automated email from the ASF dual-hosted git repository.

altay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from ddd9748  Merge pull request #4648: [BEAM-3600] Improve FileBasedSink 
rename safety
 add 1d7f684  Add missing import statements.
 new 5cf72a4  Merge pull request #4806 from udim/unittest-main

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/python/apache_beam/io/filesystem_test.py  | 1 +
 sdks/python/apache_beam/io/filesystemio_test.py| 1 +
 sdks/python/apache_beam/io/filesystems_test.py | 1 +
 sdks/python/apache_beam/io/gcp/gcsfilesystem_test.py   | 1 +
 sdks/python/apache_beam/io/hadoopfilesystem_test.py| 1 +
 sdks/python/apache_beam/io/localfilesystem_test.py | 1 +
 sdks/python/apache_beam/options/value_provider_test.py | 2 +-
 sdks/python/apache_beam/transforms/create_test.py  | 1 +
 sdks/python/apache_beam/transforms/util_test.py| 1 +
 9 files changed, 9 insertions(+), 1 deletion(-)

-- 
To stop receiving notification emails like this one, please contact
al...@apache.org.

[beam] 01/01: Merge pull request #4806 from udim/unittest-main

[altay]

This is an automated email from the ASF dual-hosted git repository.

altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 5cf72a4aee13c23c454ea0abe5a952b1f97458b0
Merge: ddd9748 1d7f684
Author: Ahmet Altay 
AuthorDate: Tue Mar 6 13:58:17 2018 -0800

Merge pull request #4806 from udim/unittest-main

Add missing import statements.

 sdks/python/apache_beam/io/filesystem_test.py  | 1 +
 sdks/python/apache_beam/io/filesystemio_test.py| 1 +
 sdks/python/apache_beam/io/filesystems_test.py | 1 +
 sdks/python/apache_beam/io/gcp/gcsfilesystem_test.py   | 1 +
 sdks/python/apache_beam/io/hadoopfilesystem_test.py| 1 +
 sdks/python/apache_beam/io/localfilesystem_test.py | 1 +
 sdks/python/apache_beam/options/value_provider_test.py | 2 +-
 sdks/python/apache_beam/transforms/create_test.py  | 1 +
 sdks/python/apache_beam/transforms/util_test.py| 1 +
 9 files changed, 9 insertions(+), 1 deletion(-)


-- 
To stop receiving notification emails like this one, please contact
al...@apache.org.

[jira] [Commented] (BEAM-3785) [SQL] Add support for arrays

[Anton Kedin (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388573#comment-16388573
 ] 

Anton Kedin commented on BEAM-3785:
---

Implemented so far:

{code:sql|title=SQL}

SELECT ARRAY ['a', 'b', 'c'] FROM PCOLLECTION;

SELECT array_field[0] FROM PCOLLECTION;

SELECT ELEMENT(array_field) FROM PCOLLECTION;

SELECT CARDINALITY(array_field) FROM PCOLLECTION;
{code}
 

> [SQL] Add support for arrays
> 
>
> Key: BEAM-3785
> URL: https://issues.apache.org/jira/browse/BEAM-3785
> Project: Beam
>  Issue Type: Improvement
>  Components: dsl-sql
>Reporter: Anton Kedin
>Assignee: Anton Kedin
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Support fields of Array type



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #91

[Apache Jenkins Server]

See 


Changes:

[yifanzou] BEAM-3339 Mobile gaming automation for Java nightly snapshot

--
[...truncated 152.64 KB...]
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (49s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (50s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (51s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (52s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (53s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (54s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (55s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (56s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (57s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (58s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (59s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (60s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (61s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (62s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (63s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (64s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (65s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (66s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (67s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (68s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (69s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (70s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (71s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (72s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (73s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (74s) Current status: 
PENDING

   
Waiting on bqjob_r5fc3728fafc4bd58_0161fd3ab2cb_1 ... (75s) Current status: 
PENDING

[jira] [Resolved] (BEAM-3600) Do not ignore FileSystem errors and document expected behavior

[Udi Meiri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Udi Meiri resolved BEAM-3600.
-
   Resolution: Fixed
Fix Version/s: 2.5.0

> Do not ignore FileSystem errors and document expected behavior
> --
>
> Key: BEAM-3600
> URL: https://issues.apache.org/jira/browse/BEAM-3600
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: Udi Meiri
>Priority: Major
> Fix For: 2.5.0
>
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> copy/rename:
>  * it should be an error if the dst file exists
>  * it should be an error if the src file doesn't exist
> delete:
>  * it should be an error if the file/dir doesn't exist
> FileBasedSink.finalize_write:
>  * should check (src, dst) pairs for existence:
>  ** src only - regular rename
>  ** dst only - skip rename
>  ** both src and dst - if files don't match on metadata (checksum or size), 
> delete dst then rename, otherwise delete src and skip rename
>  ** neither exist - return error: bad state
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1059

[Apache Jenkins Server]

See 


--
[...truncated 119.68 KB...]
self._id_to_proto[id], self._pipeline_context)
  File 
"
 line 833, in from_runner_api
transform=ptransform.PTransform.from_runner_api(proto.spec, context),
  File 
"
 line 555, in from_runner_api
context)
  File 
"
 line 886, in from_runner_api_parameter
result = ParDo(fn, *args, **kwargs)
  File 
"
 line 781, in __init__
super(ParDo, self).__init__(fn, *args, **kwargs)
  File 
"
 line 627, in __init__
self.fn = pickler.loads(pickler.dumps(self.fn))
  File 
"
 line 193, in dumps
s = dill.dumps(o)
  File 
"
 line 259, in dumps
dump(obj, file, protocol, byref, fmode, recurse)#, strictio)
  File 
"
 line 252, in dump
pik.dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
  File 
"
 line 165, in new_save_module_dict
return old_save_module_dict(pickler, obj)
  File 
"
 line 841, in save_module_dict
StockPickler.save_dict(pickler, obj)
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 675, in _batch_setitems
break
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 
"
 line 812, in run
test(orig)
  File 
"
 line 45, in __call__
return self.run(*arg, **kwarg)
  File 
"
 line 133, in run
self.runTest(result)
  File 
"
 line 151, in runTest
test(result)
  File "/usr/lib/python2.7/unittest/case.py", line 395, in __call__
return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
  File 
"
 line 178, in test_iterable_side_input
pipeline.run()
  File 
"
 line 102, in run
result = super(TestPipeline, self).run()
  File 

[jira] [Updated] (BEAM-3787) Migrate Fn API to be bidirectional instruction/request stream

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik updated BEAM-3787:

Labels: portability  (was: )

> Migrate Fn API to be bidirectional instruction/request stream
> -
>
> Key: BEAM-3787
> URL: https://issues.apache.org/jira/browse/BEAM-3787
> Project: Beam
>  Issue Type: Improvement
>  Components: beam-model
>Reporter: Luke Cwik
>Assignee: Robert Bradshaw
>Priority: Major
>  Labels: portability
>
> Allow the SDK to request the Runner to do something on its behalf. This 
> mechanism can be used for:
> * Reporting final counters
> * Work shedding (SDK can choose to reduce the amount of work it wants to do 
> (checkpointing))
> * Requesting process bundle descriptors (instead of requiring the Runner to 
> send them and have the SDK cache them).
> * Decoupling the message type in control allows for new types of messages to 
> be added which are not one to one.
> Example API change below (note that SdkMessage/RunnerMessage should use a 
> different name):
> {code:java}
> // An API that describes control messages between the SDK and Runner to 
> process
> // bundles, split bundles, report progress, ...
> service BeamFnControl {
>   // 
>   rpc Control(
> // A stream of SDK requests/responses.
> stream SdkMessage
>   ) returns (
> // A stream of Runner requests/responses.
> stream RunnerMessage
>   ) {}
> }
> // Messages a Runner can send over the control plane.
> message RunnerMessage {
>   // (Required) An unique identifier provided by the runner which represents
>   // this requests execution. The RunnerInstructionResponse MUST have the 
> matching id.
>   string id = 1;
>   oneof message {
> ErrorResponse error = 999;
> RegisterRequest register = 1000;
> ProcessBundleRequest process_bundle = 1001;
> ProcessBundleProgressRequest process_bundle_progress = 1002;
> ProcessBundleSplitRequest process_bundle_split = 1003;
> ShedBundleResponse shed_bundle = 1000;
>   }
> }
> // Messages an SDK can send over the control plane.
> message SdkMessage {
>   oneof message {
> RunnerInstructionResponse runner_instruction_response = 1000;
> SdkInstructionRequest sdk_instruction_request = 1001;
>   }
> }
> // A request sent by a runner which the SDK is asked to fulfill.
> // For any unsupported request type, an error should be returned with a
> // matching instruction id.
> // Stable
> message RunnerInstructionRequest {
>   // (Required) An unique identifier provided by the runner which represents
>   // this requests execution. The RunnerInstructionResponse MUST have the 
> matching id.
>   string instruction_id = 1;
>   // (Required) A request that the SDK Harness needs to interpret.
>   oneof request {
> RegisterRequest register = 1000;
> ProcessBundleRequest process_bundle = 1001;
> ProcessBundleProgressRequest process_bundle_progress = 1002;
> ProcessBundleSplitRequest process_bundle_split = 1003;
>   }
> }
> // The response for an associated request the SDK had been asked to fulfill.
> // Stable
> message RunnerInstructionResponse {
>   // (Required) A reference provided by the runner which represents a requests
>   // execution. The RunnerInstructionResponse MUST have the matching id when
>   // responding to the runner.
>   string instruction_id = 1;
>   // If this is specified, then this instruction has failed.
>   // A human readable string representing the reason as to why processing has
>   // failed.
>   string error = 2;
>   // If the instruction did not fail, it is required to return an equivalent
>   // response type depending on the request this matches.
>   oneof response {
> RegisterResponse register = 1000;
> ProcessBundleResponse process_bundle = 1001;
> ProcessBundleProgressResponse process_bundle_progress = 1002;
> ProcessBundleSplitResponse process_bundle_split = 1003;
>   }
> }
> message SdkInstructionRequest {
>   // (Required) An unique identifier provided by the SDK which represents
>   // this requests execution. The SdkInstructionResponse MUST have the 
> matching id.
>   string instruction_id = 1;
>   // (Required) A request that the Runner needs to interpret.
>   oneof request {
> ShedBundleRequest shed_bundle = 1000;
>   }
> }
> // The response for an associated request the Runner had been asked to 
> fulfill.
> // Stable
> message RunnerInstructionResponse {
>   // (Required) A reference provided by the SDK which represents a requests
>   // execution. The RunnerInstructionResponse MUST have the matching id when
>   // responding to the SDK.
>   string instruction_id = 1;
>   // If this is specified, then this instruction has failed.
>   // A human readable string representing the reason as to why processing has
>   // failed.

[jira] [Updated] (BEAM-3787) Migrate Fn API to be bidirectional instruction/request stream

[Luke Cwik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luke Cwik updated BEAM-3787:

Description: 
Allow the SDK to request the Runner to do something on its behalf. This 
mechanism can be used for:
* Reporting final counters
* Work shedding (SDK can choose to reduce the amount of work it wants to do 
(checkpointing))
* Requesting process bundle descriptors (instead of requiring the Runner to 
send them and have the SDK cache them).
* Decoupling the message type in control allows for new types of messages to be 
added which are not one to one.

Example API change below (note that SdkMessage/RunnerMessage should use a 
different name):

{code:java}
// An API that describes control messages between the SDK and Runner to process
// bundles, split bundles, report progress, ...
service BeamFnControl {
  // 
  rpc Control(
// A stream of SDK requests/responses.
stream SdkMessage
  ) returns (
// A stream of Runner requests/responses.
stream RunnerMessage
  ) {}
}

// Messages a Runner can send over the control plane.
message RunnerMessage {
  // (Required) An unique identifier provided by the runner which represents
  // this requests execution. The RunnerInstructionResponse MUST have the 
matching id.
  string id = 1;

  oneof message {
ErrorResponse error = 999;
RegisterRequest register = 1000;
ProcessBundleRequest process_bundle = 1001;
ProcessBundleProgressRequest process_bundle_progress = 1002;
ProcessBundleSplitRequest process_bundle_split = 1003;
ShedBundleResponse shed_bundle = 1000;
  }
}

// Messages an SDK can send over the control plane.
message SdkMessage {
  oneof message {
RunnerInstructionResponse runner_instruction_response = 1000;
SdkInstructionRequest sdk_instruction_request = 1001;
  }
}

// A request sent by a runner which the SDK is asked to fulfill.
// For any unsupported request type, an error should be returned with a
// matching instruction id.
// Stable
message RunnerInstructionRequest {
  // (Required) An unique identifier provided by the runner which represents
  // this requests execution. The RunnerInstructionResponse MUST have the 
matching id.
  string instruction_id = 1;

  // (Required) A request that the SDK Harness needs to interpret.
  oneof request {
RegisterRequest register = 1000;
ProcessBundleRequest process_bundle = 1001;
ProcessBundleProgressRequest process_bundle_progress = 1002;
ProcessBundleSplitRequest process_bundle_split = 1003;
  }
}

// The response for an associated request the SDK had been asked to fulfill.
// Stable
message RunnerInstructionResponse {
  // (Required) A reference provided by the runner which represents a requests
  // execution. The RunnerInstructionResponse MUST have the matching id when
  // responding to the runner.
  string instruction_id = 1;

  // If this is specified, then this instruction has failed.
  // A human readable string representing the reason as to why processing has
  // failed.
  string error = 2;

  // If the instruction did not fail, it is required to return an equivalent
  // response type depending on the request this matches.
  oneof response {
RegisterResponse register = 1000;
ProcessBundleResponse process_bundle = 1001;
ProcessBundleProgressResponse process_bundle_progress = 1002;
ProcessBundleSplitResponse process_bundle_split = 1003;
  }
}

message SdkInstructionRequest {
  // (Required) An unique identifier provided by the SDK which represents
  // this requests execution. The SdkInstructionResponse MUST have the matching 
id.
  string instruction_id = 1;

  // (Required) A request that the Runner needs to interpret.
  oneof request {
ShedBundleRequest shed_bundle = 1000;
  }
}

// The response for an associated request the Runner had been asked to fulfill.
// Stable
message RunnerInstructionResponse {
  // (Required) A reference provided by the SDK which represents a requests
  // execution. The RunnerInstructionResponse MUST have the matching id when
  // responding to the SDK.
  string instruction_id = 1;

  // If this is specified, then this instruction has failed.
  // A human readable string representing the reason as to why processing has
  // failed.
  string error = 2;

  // If the instruction did not fail, it is required to return an equivalent
  // response type depending on the request this matches.
  oneof response {
ShedBundleResponse shed_bundle = 1000;
  }
}

{code}


  was:
Allow the SDK to request the Runner to do something on its behalf. This 
mechanism can be used for:
* Reporting final counters
* Work shedding (SDK can choose to reduce the amount of work it wants to do 
(checkpointing))
* Requesting process bundle descriptors (instead of requiring the Runner to 
send them and have the SDK cache them).
* Decoupling the message type in control allows for new types of messages to be 
added which are not 

[jira] [Created] (BEAM-3787) Migrate Fn API to be bidirectional instruction/request stream

[Luke Cwik (JIRA)]

Luke Cwik created BEAM-3787:
---

 Summary: Migrate Fn API to be bidirectional instruction/request 
stream
 Key: BEAM-3787
 URL: https://issues.apache.org/jira/browse/BEAM-3787
 Project: Beam
  Issue Type: Improvement
  Components: beam-model
Reporter: Luke Cwik
Assignee: Robert Bradshaw


Allow the SDK to request the Runner to do something on its behalf. This 
mechanism can be used for:
* Reporting final counters
* Work shedding (SDK can choose to reduce the amount of work it wants to do 
(checkpointing))
* Requesting process bundle descriptors (instead of requiring the Runner to 
send them and have the SDK cache them).
* Decoupling the message type in control allows for new types of messages to be 
added which are not one to one.

Example API change below (note that SdkMessage/RunnerMessage should use a 
different name):

// An API that describes control messages between the SDK and Runner to process
// bundles, split bundles, report progress, ...
service BeamFnControl {
  // 
  rpc Control(
// A stream of SDK requests/responses.
stream SdkMessage
  ) returns (
// A stream of Runner requests/responses.
stream RunnerMessage
  ) {}
}

// Messages a Runner can send over the control plane.
message RunnerMessage {
  // (Required) An unique identifier provided by the runner which represents
  // this requests execution. The RunnerInstructionResponse MUST have the 
matching id.
  string id = 1;

  oneof message {
ErrorResponse error = 999;
RegisterRequest register = 1000;
ProcessBundleRequest process_bundle = 1001;
ProcessBundleProgressRequest process_bundle_progress = 1002;
ProcessBundleSplitRequest process_bundle_split = 1003;
ShedBundleResponse shed_bundle = 1000;
  }
}

// Messages an SDK can send over the control plane.
message SdkMessage {
  oneof message {
RunnerInstructionResponse runner_instruction_response = 1000;
SdkInstructionRequest sdk_instruction_request = 1001;
  }
}

// A request sent by a runner which the SDK is asked to fulfill.
// For any unsupported request type, an error should be returned with a
// matching instruction id.
// Stable
message RunnerInstructionRequest {
  // (Required) An unique identifier provided by the runner which represents
  // this requests execution. The RunnerInstructionResponse MUST have the 
matching id.
  string instruction_id = 1;

  // (Required) A request that the SDK Harness needs to interpret.
  oneof request {
RegisterRequest register = 1000;
ProcessBundleRequest process_bundle = 1001;
ProcessBundleProgressRequest process_bundle_progress = 1002;
ProcessBundleSplitRequest process_bundle_split = 1003;
  }
}

// The response for an associated request the SDK had been asked to fulfill.
// Stable
message RunnerInstructionResponse {
  // (Required) A reference provided by the runner which represents a requests
  // execution. The RunnerInstructionResponse MUST have the matching id when
  // responding to the runner.
  string instruction_id = 1;

  // If this is specified, then this instruction has failed.
  // A human readable string representing the reason as to why processing has
  // failed.
  string error = 2;

  // If the instruction did not fail, it is required to return an equivalent
  // response type depending on the request this matches.
  oneof response {
RegisterResponse register = 1000;
ProcessBundleResponse process_bundle = 1001;
ProcessBundleProgressResponse process_bundle_progress = 1002;
ProcessBundleSplitResponse process_bundle_split = 1003;
  }
}

message SdkInstructionRequest {
  // (Required) An unique identifier provided by the SDK which represents
  // this requests execution. The SdkInstructionResponse MUST have the matching 
id.
  string instruction_id = 1;

  // (Required) A request that the Runner needs to interpret.
  oneof request {
ShedBundleRequest shed_bundle = 1000;
  }
}

// The response for an associated request the Runner had been asked to fulfill.
// Stable
message RunnerInstructionResponse {
  // (Required) A reference provided by the SDK which represents a requests
  // execution. The RunnerInstructionResponse MUST have the matching id when
  // responding to the SDK.
  string instruction_id = 1;

  // If this is specified, then this instruction has failed.
  // A human readable string representing the reason as to why processing has
  // failed.
  string error = 2;

  // If the instruction did not fail, it is required to return an equivalent
  // response type depending on the request this matches.
  oneof response {
ShedBundleResponse shed_bundle = 1000;
  }
}




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Python_Verify #4373

[Apache Jenkins Server]

See 


Changes:

[1028332163] replace mockito-all

[1028332163] replace mockito

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] ignoredUnusedDeclaredDependencies

[1028332163] ban hamcrest-all and mockito-all

[1028332163] ban mockito-all and hamcrest-all

[ehudm] Improve FileBasedSink rename safety.

[lcwik] [BEAM-3690] ban mockito-all and hamcrest-all

--
[...truncated 1.02 MB...]
copying apache_beam/runners/worker/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/bundle_processor.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/data_plane_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/log_handler_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/logger_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/opcounters_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operation_specs.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.pxd -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_fast.pyx -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_slow.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/testing/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/util.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/util_test.py -> 
apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/data/standard_coders.yaml -> 
apache-beam-2.5.0.dev0/apache_beam/testing/data
copying apache_beam/testing/data/trigger_transcripts.yaml -> 
apache-beam-2.5.0.dev0/apache_beam/testing/data
copying apache_beam/transforms/__init__.py -> 
apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> 
apache-beam-2.5.0.dev0/apache_beam/transforms
copying 

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1058

[Apache Jenkins Server]

See 


Changes:

[ehudm] Improve FileBasedSink rename safety.

--
[...truncated 129.05 KB...]
return self.run(*args, **kwds)
  File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
  File 
"
 line 306, in test_flattened_side_input
pipeline.run()
  File 
"
 line 102, in run
result = super(TestPipeline, self).run()
  File 
"
 line 369, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"
 line 597, in from_runner_api
context.transforms.get_by_id(root_transform_id)]
  File 
"
 line 69, in get_by_id
self._id_to_proto[id], self._pipeline_context)
  File 
"
 line 842, in from_runner_api
part = context.transforms.get_by_id(transform_id)
  File 
"
 line 69, in get_by_id
self._id_to_proto[id], self._pipeline_context)
  File 
"
 line 842, in from_runner_api
part = context.transforms.get_by_id(transform_id)
  File 
"
 line 69, in get_by_id
self._id_to_proto[id], self._pipeline_context)
  File 
"
 line 833, in from_runner_api
transform=ptransform.PTransform.from_runner_api(proto.spec, context),
  File 
"
 line 555, in from_runner_api
context)
  File 
"
 line 886, in from_runner_api_parameter
result = ParDo(fn, *args, **kwargs)
  File 
"
 line 781, in __init__
super(ParDo, self).__init__(fn, *args, **kwargs)
  File 
"
 line 627, in __init__
self.fn = pickler.loads(pickler.dumps(self.fn))
  File 
"
 line 206, in dumps
c = zlib.compress(s, 9)
  File 
"
 line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_flattened_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)'

==
ERROR: test_iterable_side_input 
(apache_beam.transforms.sideinputs_test.SideInputsTest)
--
Traceback (most recent call last):
  File 
"
 line 812, in run
test(orig)
  File 
"
 line 45, in __call__
return self.run(*arg, **kwarg)
  File 
"
 line 133, in run
self.runTest(result)
  File 
"
 line 151, in runTest
test(result)
  File 

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #90

[Apache Jenkins Server]

See 


--
[...truncated 208.52 KB...]

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (75s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (76s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (77s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (78s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (79s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (80s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (81s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (82s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (83s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (84s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (85s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (86s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (87s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (88s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (89s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (90s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (91s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (92s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (93s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (94s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (95s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (96s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (97s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (98s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (99s) Current status: 
PENDING

   
Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (100s) Current 
status: PENDING


Waiting on bqjob_r6b96216138c1c35b_0161fcfb0b8f_1 ... (101s) Current 
status: PENDING

Build failed in Jenkins: beam_PostCommit_Python_Verify #4372

[Apache Jenkins Server]

See 


Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with 
java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite 
names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec 
packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session recording and CPU 

Build failed in Jenkins: beam_PostCommit_Python_ValidatesRunner_Dataflow #1057

[Apache Jenkins Server]

See 


Changes:

[herohde] Initial sketches of a Go SDK

[herohde] Initial version of the direct style w/ direct runner. Incomplete.

[herohde] Add Data as UserFn context w/ immediate value.

[herohde] Added no-I/O wordcount for profiling.

[herohde] Fleshed out possible approach to generic transformations.

[herohde] Add “dag” example that use multiplexing and side input.

[herohde] Added a more complex DAG example.

[herohde] Add yatzy example with more complex construction-time setup

[herohde] Add proto for Fn API

[herohde] Add beam.Composite helper for the most common pattern to align with 
java

[herohde] Move pipeline-construction time errors into an accumulator

[herohde] Add Dataflow job and Fn API clients. Incomplete.

[herohde] Add binary cross-compile and upload to Dataflow runner. Incomplete.

[herohde] Add tentative runner indirection (default: local).

[herohde] Made data flow runner detect user main for cross-compilation.

[herohde] Remove error accumulation in favor of panic.

[herohde] Improve Dataflow translation of coders, side input and composite 
names.

[herohde] Fix name for AsView.

[herohde] Use 2 grpc endpoints in harness

[herohde] Add gRPC harness logging

[herohde] Flesh out harness and serialization further.

[herohde] Made the dataflow runner wait for job termination by default

[herohde] beam:

[herohde] beam:

[herohde] combinefn.go: fix compilation issues

[herohde] Improve dataflow serialization and execution. Incomplete.

[herohde] Sleep 30 sec in wordcap to allow logs to propagate to Cloud Logging.

[herohde] Move the 30s sleep for logging to the harness instead of in WordCap.

[herohde] Post-review updates.

[herohde] Doc updates.

[herohde] Flesh out coders. Incomplete.

[herohde] Added prototype implementation of more coders and the runner source.

[herohde] dofn: illustrates how dofns are written.

[herohde] beam: add viewfn and windowfn to side inputs match support Beam 1.0

[herohde] dofn: timers

[herohde] Complete revamp: coders, graph and execution use element-wise

[herohde] Fix coder encoding for Dataflow side input. Otherwise, the job is

[herohde] Added more godoc comments to graph types.

[herohde] Added more comments plus made local GBK use coder equality.

[herohde] Added Flatten support and “forest” example that uses it.

[herohde] Move bigqueryio to defunct

[herohde] Make forest example print less

[herohde] Add external struct registry and serialization.

[herohde] Updated comments in node.go.

[herohde] Replace real type with 'full type' since that's the current term.

[herohde] Refactor Fn API dependency.

[herohde] Added more comments to the runner/dataflow and runner/beamexec 
packages

[herohde] Fix most go vet issues

[herohde] Make core operations panic to cut down on the error propagation

[herohde] Add more comments to the graph package.

[herohde] Add DoFn wrapper to handle either function or (ptr to) struct

[herohde] Fix remaining go vet warnings.

[herohde] Code review for beam/graph/coder package.

[herohde] Code review of the runtime/graphx package.

[herohde] Remove Data options in favor of using a Fn struct

[herohde] Code review of the beam/graph/userfn package.

[herohde] Code review for beam/graph package.

[herohde] godoc for runtime/graphx

[herohde] Add support for []T and Combine functions

[herohde] Add adapted documentation from the Java SDK to the beam package

[herohde] Update snapshot of Fn API.

[herohde] Add experiments flag to the Dataflow runner

[herohde] Remove context arg from beamexec.Init

[herohde] Migration to Runner API.

[herohde] Add support for creating DOT graphs.

[herohde] Make pretty printing of types and coders more concise

[herohde] Add flexible Signature to aid type checking

[herohde] Adding unit testability to harness translation.

[herohde] Fix crash due to initialization order

[herohde] Add CreateValues and Impulse

[herohde] Add Runner API support for WindowingStrategy.

[herohde] Run goimports on baseline.

[herohde] Fix encoding of global window strategy.

[herohde] Ensure the windowed value is atomically encoded.

[herohde] Limit gRPC messages to max size.

[herohde] Developer conveniences for running jobs.

[herohde] Fix sends to not close the network channel.

[herohde] Add re-iterable side input

[herohde] Add per-key Combine

[herohde] Add Min

[herohde] Reorganize non-user-facing code into core

[herohde] Make type register reject unnamed or predeclared types

[herohde] Add type specialization tool

[herohde] Don't run grpc plugin in generate phase.

[herohde] Fix import reference path for runner API proto.

[herohde] Revamp runner registration as _ imports

[herohde] Add stats.Max and Mean

[herohde] Add global pipeline options

[herohde] Unify global and per-key combiners

[herohde] Add beam convenience wrapper for imports and runner selection

[herohde] Add session 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Dataflow #5099

[Apache Jenkins Server]

See 


Changes:

[ehudm] Improve FileBasedSink rename safety.

--
Started by GitHub push by chamikaramj
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ddd9748fe950e0b274845567269c440ea7c7edeb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ddd9748fe950e0b274845567269c440ea7c7edeb
Commit message: "Merge pull request #4648: [BEAM-3600] Improve FileBasedSink 
rename safety"
 > git rev-list --no-walk 087e5117ee7cc28a5d513d81db89916b07299bd2 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 

 at hudson.remoting.Channel@4c1d78c1:beam6: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam6
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
  

Build failed in Jenkins: beam_PostCommit_Java_MavenInstall #6142

[Apache Jenkins Server]

See 


Changes:

[ehudm] Improve FileBasedSink rename safety.

--
Started by GitHub push by chamikaramj
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ddd9748fe950e0b274845567269c440ea7c7edeb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ddd9748fe950e0b274845567269c440ea7c7edeb
Commit message: "Merge pull request #4648: [BEAM-3600] Improve FileBasedSink 
rename safety"
 > git rev-list --no-walk 087e5117ee7cc28a5d513d81db89916b07299bd2 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 at 
hudson.remoting.Channel@6659890a:beam3: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam3
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Flink #5172

[Apache Jenkins Server]

See 


Changes:

[ehudm] Improve FileBasedSink rename safety.

--
Started by GitHub push by chamikaramj
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ddd9748fe950e0b274845567269c440ea7c7edeb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ddd9748fe950e0b274845567269c440ea7c7edeb
Commit message: "Merge pull request #4648: [BEAM-3600] Improve FileBasedSink 
rename safety"
 > git rev-list --no-walk 087e5117ee7cc28a5d513d81db89916b07299bd2 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@707da680:beam5: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam5
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
   

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Spark #4353

[Apache Jenkins Server]

See 


Changes:

[ehudm] Improve FileBasedSink rename safety.

--
Started by GitHub push by chamikaramj
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ddd9748fe950e0b274845567269c440ea7c7edeb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ddd9748fe950e0b274845567269c440ea7c7edeb
Commit message: "Merge pull request #4648: [BEAM-3600] Improve FileBasedSink 
rename safety"
 > git rev-list --no-walk 087e5117ee7cc28a5d513d81db89916b07299bd2 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@4c2365d7:beam8: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam8
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
at hudson.remoting.Channel.call(Channel.java:908)
at hudson.FilePath.act(FilePath.java:986)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
   

[beam] branch master updated (087e511 -> ddd9748)

[chamikara]

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 087e511  [BEAM-3690] ban mockito-all and hamcrest-all
 add 9e619a4  Improve FileBasedSink rename safety.
 new ddd9748  Merge pull request #4648: [BEAM-3600] Improve FileBasedSink 
rename safety

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../apache_beam/examples/snippets/snippets.py  |   5 +-
 sdks/python/apache_beam/io/filebasedsink.py| 181 -
 sdks/python/apache_beam/io/filebasedsink_test.py   | 149 +
 sdks/python/apache_beam/io/filesystem.py   |  22 +++
 sdks/python/apache_beam/io/filesystems.py  |  23 +++
 sdks/python/apache_beam/io/gcp/gcsfilesystem.py|  17 ++
 sdks/python/apache_beam/io/gcp/gcsio.py|  23 ++-
 sdks/python/apache_beam/io/gcp/gcsio_test.py   |  23 ++-
 sdks/python/apache_beam/io/hadoopfilesystem.py |  22 ++-
 .../python/apache_beam/io/hadoopfilesystem_test.py |  21 +++
 sdks/python/apache_beam/io/iobase.py   |  60 +--
 sdks/python/apache_beam/io/localfilesystem.py  |  16 ++
 sdks/python/apache_beam/io/localfilesystem_test.py |  10 ++
 .../transforms/write_ptransform_test.py|   6 +-
 14 files changed, 443 insertions(+), 135 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
chamik...@apache.org.

[beam] 01/01: Merge pull request #4648: [BEAM-3600] Improve FileBasedSink rename safety

[chamikara]

This is an automated email from the ASF dual-hosted git repository.

chamikara pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit ddd9748fe950e0b274845567269c440ea7c7edeb
Merge: 087e511 9e619a4
Author: Chamikara Jayalath 
AuthorDate: Tue Mar 6 11:58:17 2018 -0800

Merge pull request #4648: [BEAM-3600] Improve FileBasedSink rename safety

 .../apache_beam/examples/snippets/snippets.py  |   5 +-
 sdks/python/apache_beam/io/filebasedsink.py| 181 -
 sdks/python/apache_beam/io/filebasedsink_test.py   | 149 +
 sdks/python/apache_beam/io/filesystem.py   |  22 +++
 sdks/python/apache_beam/io/filesystems.py  |  23 +++
 sdks/python/apache_beam/io/gcp/gcsfilesystem.py|  17 ++
 sdks/python/apache_beam/io/gcp/gcsio.py|  23 ++-
 sdks/python/apache_beam/io/gcp/gcsio_test.py   |  23 ++-
 sdks/python/apache_beam/io/hadoopfilesystem.py |  22 ++-
 .../python/apache_beam/io/hadoopfilesystem_test.py |  21 +++
 sdks/python/apache_beam/io/iobase.py   |  60 +--
 sdks/python/apache_beam/io/localfilesystem.py  |  16 ++
 sdks/python/apache_beam/io/localfilesystem_test.py |  10 ++
 .../transforms/write_ptransform_test.py|   6 +-
 14 files changed, 443 insertions(+), 135 deletions(-)


-- 
To stop receiving notification emails like this one, please contact
chamik...@apache.org.

[jira] [Issue Comment Deleted] (BEAM-3665) Port BigQueryIOWriteTest off DoFnTester

[Justin Tumale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justin Tumale updated BEAM-3665:

Comment: was deleted

(was: Hi, please assign to me when you get a chance [~kenn]. I recently gained 
experience with some of the BigQuery code on my last pull request #4606 
BEAM-2817. Thanks
h1.  )

> Port BigQueryIOWriteTest off DoFnTester
> ---
>
> Key: BEAM-3665
> URL: https://issues.apache.org/jira/browse/BEAM-3665
> Project: Beam
>  Issue Type: Sub-task
>  Components: io-java-gcp
>Reporter: Kenneth Knowles
>Assignee: Justin Tumale
>Priority: Major
>  Labels: beginner, newbie, starter
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-3665) Port BigQueryIOWriteTest off DoFnTester

[Justin Tumale (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Justin Tumale reassigned BEAM-3665:
---

Assignee: Justin Tumale

> Port BigQueryIOWriteTest off DoFnTester
> ---
>
> Key: BEAM-3665
> URL: https://issues.apache.org/jira/browse/BEAM-3665
> Project: Beam
>  Issue Type: Sub-task
>  Components: io-java-gcp
>Reporter: Kenneth Knowles
>Assignee: Justin Tumale
>Priority: Major
>  Labels: beginner, newbie, starter
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (BEAM-3665) Port BigQueryIOWriteTest off DoFnTester

[Justin Tumale (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-3665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388421#comment-16388421
 ] 

Justin Tumale edited comment on BEAM-3665 at 3/6/18 7:55 PM:
-

Hi, please assign to me when you get a chance [~kenn]. I recently gained 
experience with some of the BigQuery code on my last pull request #4606 
BEAM-2817. Thanks
h1.  


was (Author: justintumale):
Hi, please assign to me when you get a chance [~kenn]. Thanks

> Port BigQueryIOWriteTest off DoFnTester
> ---
>
> Key: BEAM-3665
> URL: https://issues.apache.org/jira/browse/BEAM-3665
> Project: Beam
>  Issue Type: Sub-task
>  Components: io-java-gcp
>Reporter: Kenneth Knowles
>Priority: Major
>  Labels: beginner, newbie, starter
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Apex #3632

[Apache Jenkins Server]

See 


Changes:

[1028332163] replace mockito-all

[1028332163] replace mockito

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] ignoredUnusedDeclaredDependencies

[1028332163] ban hamcrest-all and mockito-all

[1028332163] ban mockito-all and hamcrest-all

[lcwik] [BEAM-3690] ban mockito-all and hamcrest-all

--
Started by GitHub push by asfgit
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 087e5117ee7cc28a5d513d81db89916b07299bd2 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 087e5117ee7cc28a5d513d81db89916b07299bd2
Commit message: "[BEAM-3690] ban mockito-all and hamcrest-all"
 > git rev-list --no-walk e3e4e3ce31b55f8cc403aa2b376814a1130ff49c # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@6659890a:beam3: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam3
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Dataflow #5098

[Apache Jenkins Server]

See 


Changes:

[1028332163] replace mockito-all

[1028332163] replace mockito

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] ignoredUnusedDeclaredDependencies

[1028332163] ban hamcrest-all and mockito-all

[1028332163] ban mockito-all and hamcrest-all

[lcwik] [BEAM-3690] ban mockito-all and hamcrest-all

--
Started by GitHub push by asfgit
[EnvInject] - Loading node environment variables.
Building remotely on beam5 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 087e5117ee7cc28a5d513d81db89916b07299bd2 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 087e5117ee7cc28a5d513d81db89916b07299bd2
Commit message: "[BEAM-3690] ban mockito-all and hamcrest-all"
 > git rev-list --no-walk e3e4e3ce31b55f8cc403aa2b376814a1130ff49c # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 

 at hudson.remoting.Channel@707da680:beam5: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam5
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)

Jenkins build is back to normal : beam_PostRelease_NightlySnapshot #89

[Apache Jenkins Server]

See 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Flink #5171

[Apache Jenkins Server]

See 


Changes:

[1028332163] replace mockito-all

[1028332163] replace mockito

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] ignoredUnusedDeclaredDependencies

[1028332163] ban hamcrest-all and mockito-all

[1028332163] ban mockito-all and hamcrest-all

[lcwik] [BEAM-3690] ban mockito-all and hamcrest-all

--
Started by GitHub push by asfgit
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 087e5117ee7cc28a5d513d81db89916b07299bd2 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 087e5117ee7cc28a5d513d81db89916b07299bd2
Commit message: "[BEAM-3690] ban mockito-all and hamcrest-all"
 > git rev-list --no-walk e3e4e3ce31b55f8cc403aa2b376814a1130ff49c # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@a7114c0:beam4: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam4
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at 

Build failed in Jenkins: beam_PostCommit_Java_ValidatesRunner_Spark #4352

[Apache Jenkins Server]

See 


Changes:

[1028332163] replace mockito-all

[1028332163] replace mockito

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] replace mockito-all and harcrest-all

[1028332163] ignoredUnusedDeclaredDependencies

[1028332163] ban hamcrest-all and mockito-all

[1028332163] ban mockito-all and hamcrest-all

[lcwik] [BEAM-3690] ban mockito-all and hamcrest-all

--
Started by GitHub push by asfgit
[EnvInject] - Loading node environment variables.
Building remotely on beam3 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 087e5117ee7cc28a5d513d81db89916b07299bd2 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 087e5117ee7cc28a5d513d81db89916b07299bd2
Commit message: "[BEAM-3690] ban mockito-all and hamcrest-all"
 > git rev-list --no-walk e3e4e3ce31b55f8cc403aa2b376814a1130ff49c # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
Parsing POMs
ERROR: Failed to parse POMs
java.io.IOException: remote file operation failed: 
 
at hudson.remoting.Channel@6659890a:beam3: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at hudson.FilePath.act(FilePath.java:993)
at hudson.FilePath.act(FilePath.java:975)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.parsePoms(MavenModuleSetBuild.java:986)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:691)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: hudson.remoting.ProxyException: 
hudson.maven.MavenModuleSetBuild$MavenExecutionException: 
org.apache.maven.project.ProjectBuildingException: Some problems were 
encountered while processing the POMs:
[FATAL] Non-resolvable parent POM: Could not find artifact 
org.apache.beam:beam-sdks-go:pom:2.3.0-SNAPSHOT and 'parent.relativePath' 
points at wrong local POM @ line 21, column 11

at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1391)
at 
hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1125)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2760)
at hudson.remoting.UserRequest.perform(UserRequest.java:207)
at hudson.remoting.UserRequest.perform(UserRequest.java:53)
at hudson.remoting.Request$2.run(Request.java:358)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to 
beam3
at 
hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1693)
at 

[beam] 03/03: [BEAM-3690] ban mockito-all and hamcrest-all

[lcwik]

This is an automated email from the ASF dual-hosted git repository.

lcwik pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 087e5117ee7cc28a5d513d81db89916b07299bd2
Merge: e3e4e3c 91ca61d
Author: Luke Cwik 
AuthorDate: Tue Mar 6 11:51:24 2018 -0800

[BEAM-3690] ban mockito-all and hamcrest-all

This closes #4712

 pom.xml|  2 ++
 .../elasticsearch-tests-5/pom.xml  | 22 ++
 sdks/java/nexmark/pom.xml  |  1 +
 3 files changed, 9 insertions(+), 16 deletions(-)

-- 
To stop receiving notification emails like this one, please contact
lc...@apache.org.