[jira] [Commented] (BEAM-5050) [SQL] NULLs are aggregated incorrectly

[Gleb Kanterov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16564915#comment-16564915
 ] 

Gleb Kanterov commented on BEAM-5050:
-

Hey [~kedin], you can assign it to me :)

> [SQL] NULLs are aggregated incorrectly
> --
>
> Key: BEAM-5050
> URL: https://issues.apache.org/jira/browse/BEAM-5050
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Anton Kedin
>Priority: Major
>
> For example, COUNT(field) should not count records with NULL field. We also 
> should handle and test on other aggregation functions (like AVG, SUM, MIN, 
> MAX, VAR_POP, VAR_SAMP, etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4842) Update Flink Runner to Flink 1.5.2

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4842?focusedWorklogId=129609&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129609
 ]

ASF GitHub Bot logged work on BEAM-4842:


Author: ASF GitHub Bot
Created on: 01/Aug/18 07:57
Start Date: 01/Aug/18 07:57
Worklog Time Spent: 10m 
  Work Description: iemejia commented on issue #6109: [BEAM-4842] Update 
Flink Runner to Flink 1.5.2
URL: https://github.com/apache/beam/pull/6109#issuecomment-409485383
 
 
   Run Flink ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129609)
Time Spent: 2h 20m  (was: 2h 10m)

> Update Flink Runner to Flink 1.5.2
> --
>
> Key: BEAM-4842
> URL: https://issues.apache.org/jira/browse/BEAM-4842
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4842) Update Flink Runner to Flink 1.5.2

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4842?focusedWorklogId=129610&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129610
 ]

ASF GitHub Bot logged work on BEAM-4842:


Author: ASF GitHub Bot
Created on: 01/Aug/18 07:57
Start Date: 01/Aug/18 07:57
Worklog Time Spent: 10m 
  Work Description: iemejia commented on issue #6109: [BEAM-4842] Update 
Flink Runner to Flink 1.5.2
URL: https://github.com/apache/beam/pull/6109#issuecomment-409485484
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129610)
Time Spent: 2.5h  (was: 2h 20m)

> Update Flink Runner to Flink 1.5.2
> --
>
> Key: BEAM-4842
> URL: https://issues.apache.org/jira/browse/BEAM-4842
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4686) Build Failure: gpg: keyserver receive failed: unknown pubkey algorithm

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4686?focusedWorklogId=129620&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129620
 ]

ASF GitHub Bot logged work on BEAM-4686:


Author: ASF GitHub Bot
Created on: 01/Aug/18 08:29
Start Date: 01/Aug/18 08:29
Worklog Time Spent: 10m 
  Work Description: jasonkuster commented on a change in pull request 
#6111: [BEAM-4686] Add retries for gpg key query logic.
URL: https://github.com/apache/beam/pull/6111#discussion_r206794877
 
 

 ##
 File path: .test-infra/jenkins/job_PreCommit_Website_Merge.groovy
 ##
 @@ -17,6 +17,7 @@
  */
 
 import CommonJobProperties as commonJobProperties
 
 Review comment:
   Ok cool, sounds fine.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129620)
Time Spent: 1h 20m  (was: 1h 10m)

> Build Failure: gpg: keyserver receive failed: unknown pubkey algorithm
> --
>
> Key: BEAM-4686
> URL: https://issues.apache.org/jira/browse/BEAM-4686
> Project: Beam
>  Issue Type: Bug
>  Components: website
>Affects Versions: 2.6.0
>Reporter: Andrew Pilloud
>Assignee: Mikhail Gryzykhin
>Priority: Major
>  Labels: flake
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> I see this on a large percentage of builds:
>  
> https://builds.apache.org/job/beam_PreCommit_Website_Stage/1145/console
> {code:java}
> [beam_PreCommit_Website_Stage] $ /bin/bash -xe 
> /tmp/jenkins6730031359263725887.sh
> + gpg --keyserver hkp://keys.gnupg.net --recv-keys 
> 409B6B1796C275462A1703113804BB82D39DC0E3 
> 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
> gpg: requesting key D39DC0E3 from hkp server keys.gnupg.net
> gpg: requesting key 39499BDB from hkp server keys.gnupg.net
> gpgkeys: key 409B6B1796C275462A1703113804BB82D39DC0E3 can't be retrieved
> gpgkeys: key 7D2BAF1CF37B13E2069D6956105BD0E739499BDB can't be retrieved
> gpg: no valid OpenPGP data found.
> gpg: Total number processed: 0
> gpg: keyserver communications error: keyserver helper general error
> gpg: keyserver communications error: unknown pubkey algorithm
> gpg: keyserver receive failed: unknown pubkey algorithm
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated (ddef7de -> 435ea78)

[jaku]

This is an automated email from the ASF dual-hosted git repository.

jaku pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from ddef7de  Merge pull request #6114: [BEAM-4453] Add convenience methods 
for pojo and javabean schema registration.
 add 0306e80  Add retries for gpg key query logic.
 new 435ea78  Merge pull request #6111: [BEAM-4686] Add retries for gpg key 
query logic

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 ...t_Website_Merge.groovy => WebsiteShared.groovy} | 45 ++
 .../jenkins/job_PreCommit_Website_Merge.groovy | 29 ++
 .../jenkins/job_PreCommit_Website_Stage.groovy | 36 -
 .../jenkins/job_PreCommit_Website_Test.groovy  | 29 ++
 4 files changed, 36 insertions(+), 103 deletions(-)
 copy .test-infra/jenkins/{job_PreCommit_Website_Merge.groovy => 
WebsiteShared.groovy} (64%)
 mode change 100644 => 100755

[beam] 01/01: Merge pull request #6111: [BEAM-4686] Add retries for gpg key query logic

[jaku]

This is an automated email from the ASF dual-hosted git repository.

jaku pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 435ea784ed73a75435f9711ede1eaae1c7319270
Merge: ddef7de 0306e80
Author: jasonkuster 
AuthorDate: Wed Aug 1 01:31:31 2018 -0700

Merge pull request #6111: [BEAM-4686] Add retries for gpg key query logic

[BEAM-4686] Add retries for gpg key query logic.

 ...t_Website_Merge.groovy => WebsiteShared.groovy} | 45 ++
 .../jenkins/job_PreCommit_Website_Merge.groovy | 29 ++
 .../jenkins/job_PreCommit_Website_Stage.groovy | 36 -
 .../jenkins/job_PreCommit_Website_Test.groovy  | 29 ++
 4 files changed, 36 insertions(+), 103 deletions(-)

[jira] [Work logged] (BEAM-4686) Build Failure: gpg: keyserver receive failed: unknown pubkey algorithm

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4686?focusedWorklogId=129622&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129622
 ]

ASF GitHub Bot logged work on BEAM-4686:


Author: ASF GitHub Bot
Created on: 01/Aug/18 08:31
Start Date: 01/Aug/18 08:31
Worklog Time Spent: 10m 
  Work Description: jasonkuster closed pull request #6111: [BEAM-4686] Add 
retries for gpg key query logic.
URL: https://github.com/apache/beam/pull/6111
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/WebsiteShared.groovy 
b/.test-infra/jenkins/WebsiteShared.groovy
new file mode 100755
index 000..ff2b733b5bb
--- /dev/null
+++ b/.test-infra/jenkins/WebsiteShared.groovy
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+@groovy.transform.Field static final String install_ruby_and_gems_bash = '''
+maxKeyFetchAttempts=5
+currentAttempt=0
+retryDelaySeconds=15
+
+while [ $currentAttempt -lt $maxKeyFetchAttempts ]
+do
+  echo "Attempt #$currentAttempt"
+  gpg --keyserver hkp://keys.gnupg.net --recv-keys 
409B6B1796C275462A1703113804BB82D39DC0E3 
7D2BAF1CF37B13E2069D6956105BD0E739499BDB && break
+  echo "Attempt failed. Sleeping $retryDelaySeconds seconds, then 
retrying"
+  n=$((n+1))
+  sleep $retryDelaySeconds
+done
+
+if [ $currentAttempt -ge $maxKeyFetchAttempts ]
+then
+  echo "ERROR: Failed to fetch gpg keys."
+  exit 1
+fi
+
+\\curl -sSL https://get.rvm.io | bash
+source /home/jenkins/.rvm/scripts/rvm
+
+# Install Ruby.
+RUBY_VERSION_NUM=2.3.0
+rvm install ruby $RUBY_VERSION_NUM --autolibs=read-only
+
+# Install Bundler gem
+PATH=~/.gem/ruby/$RUBY_VERSION_NUM/bin:$PATH
+GEM_PATH=~/.gem/ruby/$RUBY_VERSION_NUM/:$GEM_PATH
+gem install bundler --user-install
+
+# Enter the git clone for remaining commands
+cd src
+
+# Install all needed gems.
+bundle install --path ~/.gem/
+
+'''
diff --git a/.test-infra/jenkins/job_PreCommit_Website_Merge.groovy 
b/.test-infra/jenkins/job_PreCommit_Website_Merge.groovy
index 235c728bb83..d65fdb89fff 100644
--- a/.test-infra/jenkins/job_PreCommit_Website_Merge.groovy
+++ b/.test-infra/jenkins/job_PreCommit_Website_Merge.groovy
@@ -17,6 +17,7 @@
  */
 
 import CommonJobProperties as commonJobProperties
+import WebsiteShared as websiteShared
 
 // Defines a job.
 job('beam_PreCommit_Website_Merge') {
@@ -31,33 +32,11 @@ job('beam_PreCommit_Website_Merge') {
 
   steps {
 // Run the following shell script as a build step.
-shell '''
-# Install RVM per instructions at https://rvm.io/rvm/install.
-gpg --keyserver hkp://keys.gnupg.net --recv-keys \\
-409B6B1796C275462A1703113804BB82D39DC0E3 \\
-7D2BAF1CF37B13E2069D6956105BD0E739499BDB
-
-\\curl -sSL https://get.rvm.io | bash
-source /home/jenkins/.rvm/scripts/rvm
-
-# Install Ruby.
-RUBY_VERSION_NUM=2.3.0
-rvm install ruby $RUBY_VERSION_NUM --autolibs=read-only
-
-# Install Bundler gem
-PATH=~/.gem/ruby/$RUBY_VERSION_NUM/bin:$PATH
-GEM_PATH=~/.gem/ruby/$RUBY_VERSION_NUM/:$GEM_PATH
-gem install bundler --user-install
-
-# Enter the git clone for remaining commands
-cd src
-
-# Install all needed gems.
-bundle install --path ~/.gem/
-
+shell """
+${websiteShared.install_ruby_and_gems_bash}
 # Build the new site and test it.
 rm -fr ./content/
 bundle exec rake test
-'''.stripIndent().trim()
+""".stripIndent().trim()
   }
 }
diff --git a/.test-infra/jenkins/job_PreCommit_Website_Stage.groovy 
b/.test-infra/jenkins/job_PreCommit_Website_Stage.groovy
index 50e1e3e5c

[jira] [Work logged] (BEAM-4686) Build Failure: gpg: keyserver receive failed: unknown pubkey algorithm

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4686?focusedWorklogId=129621&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129621
 ]

ASF GitHub Bot logged work on BEAM-4686:


Author: ASF GitHub Bot
Created on: 01/Aug/18 08:31
Start Date: 01/Aug/18 08:31
Worklog Time Spent: 10m 
  Work Description: jasonkuster commented on issue #6111: [BEAM-4686] Add 
retries for gpg key query logic.
URL: https://github.com/apache/beam/pull/6111#issuecomment-409495243
 
 
   Took a look at the seed job and it worked; LGTM. Merging...


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129621)
Time Spent: 1.5h  (was: 1h 20m)

> Build Failure: gpg: keyserver receive failed: unknown pubkey algorithm
> --
>
> Key: BEAM-4686
> URL: https://issues.apache.org/jira/browse/BEAM-4686
> Project: Beam
>  Issue Type: Bug
>  Components: website
>Affects Versions: 2.6.0
>Reporter: Andrew Pilloud
>Assignee: Mikhail Gryzykhin
>Priority: Major
>  Labels: flake
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> I see this on a large percentage of builds:
>  
> https://builds.apache.org/job/beam_PreCommit_Website_Stage/1145/console
> {code:java}
> [beam_PreCommit_Website_Stage] $ /bin/bash -xe 
> /tmp/jenkins6730031359263725887.sh
> + gpg --keyserver hkp://keys.gnupg.net --recv-keys 
> 409B6B1796C275462A1703113804BB82D39DC0E3 
> 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
> gpg: requesting key D39DC0E3 from hkp server keys.gnupg.net
> gpg: requesting key 39499BDB from hkp server keys.gnupg.net
> gpgkeys: key 409B6B1796C275462A1703113804BB82D39DC0E3 can't be retrieved
> gpgkeys: key 7D2BAF1CF37B13E2069D6956105BD0E739499BDB can't be retrieved
> gpg: no valid OpenPGP data found.
> gpg: Total number processed: 0
> gpg: keyserver communications error: keyserver helper general error
> gpg: keyserver communications error: unknown pubkey algorithm
> gpg: keyserver receive failed: unknown pubkey algorithm
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5056) [SQL] Nullability of aggregation expressions isn't inferred properly

[Gleb Kanterov (JIRA)]

Gleb Kanterov created BEAM-5056:
---

 Summary: [SQL] Nullability of aggregation expressions isn't 
inferred properly
 Key: BEAM-5056
 URL: https://issues.apache.org/jira/browse/BEAM-5056
 Project: Beam
  Issue Type: Bug
  Components: dsl-sql
Reporter: Gleb Kanterov
Assignee: Xu Mingmin


Given schema and rows:
{code:java}
Schema schema =
Schema.builder()
.addNullableField("f_int1", Schema.FieldType.INT32)
.addNullableField("f_int2", Schema.FieldType.INT32)
.build();

List rows =
TestUtils.RowsBuilder.of(schema)
.addRows(null, null)
.getRows();
{code}
Following query fails:
{code:sql}
SELECT AVG(f_int1) FROM PCOLLECTION GROUP BY f_int2
{code}
{code:java}
Caused by: java.lang.IllegalArgumentException: Field EXPR$0 is not 
nullable{code}
 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5056) [SQL] Nullability of aggregation expressions isn't inferred properly

[Gleb Kanterov (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16565011#comment-16565011
 ] 

Gleb Kanterov commented on BEAM-5056:
-

I think I have a fix for this issue, problem is in  `BeamAggregationTransforms`

> [SQL] Nullability of aggregation expressions isn't inferred properly
> 
>
> Key: BEAM-5056
> URL: https://issues.apache.org/jira/browse/BEAM-5056
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Gleb Kanterov
>Assignee: Xu Mingmin
>Priority: Major
>
> Given schema and rows:
> {code:java}
> Schema schema =
> Schema.builder()
> .addNullableField("f_int1", Schema.FieldType.INT32)
> .addNullableField("f_int2", Schema.FieldType.INT32)
> .build();
> List rows =
> TestUtils.RowsBuilder.of(schema)
> .addRows(null, null)
> .getRows();
> {code}
> Following query fails:
> {code:sql}
> SELECT AVG(f_int1) FROM PCOLLECTION GROUP BY f_int2
> {code}
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Field EXPR$0 is not 
> nullable{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5050) [SQL] NULLs are aggregated incorrectly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5050?focusedWorklogId=129625&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129625
 ]

ASF GitHub Bot logged work on BEAM-5050:


Author: ASF GitHub Bot
Created on: 01/Aug/18 09:00
Start Date: 01/Aug/18 09:00
Worklog Time Spent: 10m 
  Work Description: kanterov commented on a change in pull request #6108: 
[BEAM-5050] [SQL] Fix aggregation of nulls
URL: https://github.com/apache/beam/pull/6108#discussion_r206803788
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslAggregationNullableTest.java
 ##
 @@ -0,0 +1,156 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.extensions.sql;
+
+import static 
org.apache.beam.sdk.extensions.sql.utils.RowAsserts.matchesScalar;
+
+import java.util.List;
+import org.apache.beam.sdk.schemas.Schema;
+import org.apache.beam.sdk.testing.PAssert;
+import org.apache.beam.sdk.testing.TestPipeline;
+import org.apache.beam.sdk.transforms.Create;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.Row;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Rule;
+import org.junit.Test;
+
+/** Integration tests for aggregation nullable columns. */
+public class BeamSqlDslAggregationNullableTest {
+
+  @Rule public TestPipeline pipeline = TestPipeline.create();
+
+  private PCollection boundedInput;
+
+  @Before
+  public void setUp() {
+Schema schema =
+Schema.builder()
+.addNullableField("f_int1", Schema.FieldType.INT32)
+.addNullableField("f_int2", Schema.FieldType.INT32)
+.addInt32Field("f_int3")
+.build();
+
+List rows =
+TestUtils.RowsBuilder.of(schema)
+.addRows(1, 5, 1)
+.addRows(null, 1, 1)
+.addRows(2, 1, 1)
+.addRows(null, 1, 1)
+.addRows(null, null, 1)
+.addRows(null, null, 1)
+.addRows(3, 2, 1)
+.getRows();
+
+boundedInput = 
PBegin.in(pipeline).apply(Create.of(rows).withCoder(schema.getRowCoder()));
+  }
+
+  @Test
+  public void testCount() {
+String sql = "SELECT COUNT(f_int1) FROM PCOLLECTION GROUP BY f_int3";
+
+
PAssert.that(boundedInput.apply(SqlTransform.query(sql))).satisfies(matchesScalar(3L));
+
+pipeline.run();
+  }
+
+  @Test
+  public void testCountStar() {
+String sql = "SELECT COUNT(*) FROM PCOLLECTION GROUP BY f_int3";
+
+
PAssert.that(boundedInput.apply(SqlTransform.query(sql))).satisfies(matchesScalar(7L));
+
+pipeline.run();
+  }
+
+  @Test
+  public void testCountThroughSum() {
+String sql =
+"SELECT SUM(CASE f_int1 IS NOT NULL WHEN TRUE THEN 1 ELSE 0 END) "
++ "FROM PCOLLECTION GROUP BY f_int3";
+
+
PAssert.that(boundedInput.apply(SqlTransform.query(sql))).satisfies(matchesScalar(3));
+
+pipeline.run();
+  }
+
+  @Test
+  public void testCountNulls() {
+String sql =
+"SELECT SUM(CASE f_int1 IS NULL WHEN TRUE THEN 1 ELSE 0 END) "
++ "FROM PCOLLECTION GROUP BY f_int3";
+
+
PAssert.that(boundedInput.apply(SqlTransform.query(sql))).satisfies(matchesScalar(4));
+
+pipeline.run();
+  }
+
+  @Test
+  public void testSum() {
+String sql = "SELECT SUM(f_int1) FROM PCOLLECTION GROUP BY f_int3";
+
+
PAssert.that(boundedInput.apply(SqlTransform.query(sql))).satisfies(matchesScalar(6));
+
+pipeline.run();
+  }
+
+  @Test
+  public void testAvg() {
+String sql = "SELECT AVG(f_int1) FROM PCOLLECTION GROUP BY f_int3";
+
+
PAssert.that(boundedInput.apply(SqlTransform.query(sql))).satisfies(matchesScalar(2));
+
+pipeline.run();
+  }
+
+  @Ignore
+  // FIXME java.lang.IllegalArgumentException: Field EXPR$0 is not nullable
 
 Review comment:
   Created a ticket: https://issues.apache.org/jira/browse/BEAM-5056


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHu

[jira] [Work logged] (BEAM-5050) [SQL] NULLs are aggregated incorrectly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5050?focusedWorklogId=129627&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129627
 ]

ASF GitHub Bot logged work on BEAM-5050:


Author: ASF GitHub Bot
Created on: 01/Aug/18 09:03
Start Date: 01/Aug/18 09:03
Worklog Time Spent: 10m 
  Work Description: kanterov commented on issue #6108: [BEAM-5050] [SQL] 
Fix aggregation of nulls
URL: https://github.com/apache/beam/pull/6108#issuecomment-409505308
 
 
   @XuMingmin rebased, thanks!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129627)
Time Spent: 20m  (was: 10m)

> [SQL] NULLs are aggregated incorrectly
> --
>
> Key: BEAM-5050
> URL: https://issues.apache.org/jira/browse/BEAM-5050
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Anton Kedin
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> For example, COUNT(field) should not count records with NULL field. We also 
> should handle and test on other aggregation functions (like AVG, SUM, MIN, 
> MAX, VAR_POP, VAR_SAMP, etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : beam_PostCommit_Java_GradleBuild #1139

[Apache Jenkins Server]

See 

[beam] 01/01: Merge pull request #6109: [BEAM-4842] Update Flink Runner to Flink 1.5.2

[iemejia]

This is an automated email from the ASF dual-hosted git repository.

iemejia pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Merge: 435ea78 65cbe3b
Author: Ismaël Mejía 
AuthorDate: Wed Aug 1 11:29:40 2018 +0200

Merge pull request #6109: [BEAM-4842] Update Flink Runner to Flink 1.5.2

 runners/flink/build.gradle | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[jira] [Resolved] (BEAM-4842) Update Flink Runner to Flink 1.5.2

[JIRA]

 [ 
https://issues.apache.org/jira/browse/BEAM-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ismaël Mejía resolved BEAM-4842.

Resolution: Fixed

> Update Flink Runner to Flink 1.5.2
> --
>
> Key: BEAM-4842
> URL: https://issues.apache.org/jira/browse/BEAM-4842
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] branch master updated (435ea78 -> f2ae73b)

[iemejia]

This is an automated email from the ASF dual-hosted git repository.

iemejia pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 435ea78  Merge pull request #6111: [BEAM-4686] Add retries for gpg key 
query logic
 add 65cbe3b  [BEAM-4842] Update Flink Runner to Flink 1.5.2
 new f2ae73b  Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 runners/flink/build.gradle | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

[jira] [Work logged] (BEAM-4842) Update Flink Runner to Flink 1.5.2

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4842?focusedWorklogId=129634&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129634
 ]

ASF GitHub Bot logged work on BEAM-4842:


Author: ASF GitHub Bot
Created on: 01/Aug/18 09:29
Start Date: 01/Aug/18 09:29
Worklog Time Spent: 10m 
  Work Description: iemejia closed pull request #6109: [BEAM-4842] Update 
Flink Runner to Flink 1.5.2
URL: https://github.com/apache/beam/pull/6109
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/runners/flink/build.gradle b/runners/flink/build.gradle
index de51c2affed..2b255d2f227 100644
--- a/runners/flink/build.gradle
+++ b/runners/flink/build.gradle
@@ -46,7 +46,7 @@ configurations {
   validatesRunner
 }
 
-def flink_version = "1.5.1"
+def flink_version = "1.5.2"
 
 dependencies {
   compile library.java.guava


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129634)
Time Spent: 2h 40m  (was: 2.5h)

> Update Flink Runner to Flink 1.5.2
> --
>
> Key: BEAM-4842
> URL: https://issues.apache.org/jira/browse/BEAM-4842
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Major
> Fix For: 2.7.0
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129635&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129635
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 09:30
Start Date: 01/Aug/18 09:30
Worklog Time Spent: 10m 
  Work Description: iemejia commented on issue #6101: [BEAM-4828] Add 
Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#issuecomment-409513545
 
 
   Retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129635)
Time Spent: 3h 40m  (was: 3.5h)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4835) Add more flexible options for data loading to BigQueryIO.Write

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4835?focusedWorklogId=129640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129640
 ]

ASF GitHub Bot logged work on BEAM-4835:


Author: ASF GitHub Bot
Created on: 01/Aug/18 09:59
Start Date: 01/Aug/18 09:59
Worklog Time Spent: 10m 
  Work Description: gene-telligent commented on issue #6008: [BEAM-4835] 
Adding in additional options for BigQueryIO insert statements
URL: https://github.com/apache/beam/pull/6008#issuecomment-409521605
 
 
   @lukecwik Rebased and ready to merge.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129640)
Time Spent: 0.5h  (was: 20m)

> Add more flexible options for data loading to BigQueryIO.Write
> --
>
> Key: BEAM-4835
> URL: https://issues.apache.org/jira/browse/BEAM-4835
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Reporter: Gene Peters
>Assignee: Chamikara Jayalath
>Priority: Minor
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> As part of the BigQuery API, there are a few options exposed to end-users 
> which allow for more flexible data loading.
> For both 
> [streaming|https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/TableDataInsertAllRequest.html#setIgnoreUnknownValues-java.lang.Boolean-]
>  and 
> [batch|https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/JobConfigurationLoad.html#setIgnoreUnknownValues-java.lang.Boolean-]
>  inserts, the flag "ignoreUnknownValues" can be set, which indicates if 
> BigQuery should accept rows that contain values that do not match the schema. 
> [In 
> addition,|https://developers.google.com/resources/api-libraries/documentation/bigquery/v2/java/latest/com/google/api/services/bigquery/model/TableDataInsertAllRequest.html#setSkipInvalidRows-java.lang.Boolean-]
>  streaming inserts allow for the option of accepting an inserted batch of 
> rows even if some of of the rows are invalid.
> I've made the necessary code changes to make this available within 
> BigQueryIO.Write and will be attaching the pull request to this ticket for 
> review. Both flags are off by default.
> Let me know if you have any questions or feedback about this!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4864) run_validatescontainer.sh in Python sdk has hard-coded bucket of sadness

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4864?focusedWorklogId=129665&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129665
 ]

ASF GitHub Bot logged work on BEAM-4864:


Author: ASF GitHub Bot
Created on: 01/Aug/18 11:58
Start Date: 01/Aug/18 11:58
Worklog Time Spent: 10m 
  Work Description: alanmyrvold commented on a change in pull request 
#6061: [BEAM-4864] Make run_validatescontainer.sh not quite as hard coded.
URL: https://github.com/apache/beam/pull/6061#discussion_r206851053
 
 

 ##
 File path: sdks/python/container/run_validatescontainer.sh
 ##
 @@ -25,14 +25,16 @@
 #
 # Execute from the root of the repository: 
sdks/python/container/run_validatescontainer.sh
 
+echo "This script must be executed in the root of beam project. Please set 
LOCAL_PATH, GCS_LCOATION, and PROJECT as desired."
 
 Review comment:
   GCS_LOCATION


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129665)
Time Spent: 50m  (was: 40m)

> run_validatescontainer.sh in Python sdk has hard-coded bucket of sadness
> 
>
> Key: BEAM-4864
> URL: https://issues.apache.org/jira/browse/BEAM-4864
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness
>Reporter: holdenk
>Assignee: holdenk
>Priority: Trivial
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> The run_validatescontainer.sh script looks amazing! However I could not 
> validate my container, and this made me sad. We can make it configurable and 
> then people can validate their container changes more easily :)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: beam_PerformanceTests_JDBC #921

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam11 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins6942767047697489737.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins8621115376203669436.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins1947938714845350710.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-jdbc-921
namespace "beam-performancetests-jdbc-921" created
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins4727717079920828031.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-jdbc-921
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins8279065812944762768.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins6338493502285894093.sh
+ rm -rf .beam_env
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins6073050871759980217.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins7630782463664631321.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins5780148446976135345.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins6808212474099864880.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_JDBC] $ /bin/bash -xe /tmp/jenkins225759539550742465.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_en

Build failed in Jenkins: beam_PerformanceTests_AvroIOIT_HDFS #489

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins8763912408600054123.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7991301987930381364.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins8215412367892602318.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-avroioit-hdfs-489
namespace "beam-performancetests-avroioit-hdfs-489" created
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3337812170634982732.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-avroioit-hdfs-489
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins6768067778568451665.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3025843495993320023.sh
+ rm -rf .beam_env
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins9190810317699256951.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins2169937579850965346.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3600874690643697265.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins1600032766792268290.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe

Build failed in Jenkins: beam_PerformanceTests_XmlIOIT #585

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins3620314713406556216.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins213951458826264868.sh
+ rm -rf .beam_env
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins5105512017722093569.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins1669655661520210472.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins5193369322755159090.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins856008126673806830.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins5407780184750952287.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins1102418145788372900.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_XmlIOIT] $ /bin/bash -xe 
/tmp/jenkins5755241372496913086.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-b

Build failed in Jenkins: beam_PerformanceTests_Spark #2028

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam2 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins5557174967754291018.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins1692762795076252957.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins48981217645796355.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins584660971487787.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins2327770582740354231.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins6944629777439923225.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins5265588769331908766.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins4710005276013788421.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Spark] $ /bin/bash -xe /tmp/jenkins3580171319092821173.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.0.0)
Requirement al

Build failed in Jenkins: beam_PerformanceTests_HadoopInputFormat #590

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam8 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins5415656468384087490.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins9158565131898054402.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins1116356753512068874.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-hadoopinputformat-590
namespace "beam-performancetests-hadoopinputformat-590" created
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins1883734831510831777.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-hadoopinputformat-590
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins6905698037851133091.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins1211570295930164782.sh
+ rm -rf .beam_env
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins7067755355299646717.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins490288839019907314.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins4357872378348479511.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe 
/tmp/jenkins8171329280717644712.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./

Build failed in Jenkins: beam_PerformanceTests_TextIOIT_HDFS #500

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam11 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7982781515727404067.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins519221136874774749.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7750186293612131884.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-textioit-hdfs-500
namespace "beam-performancetests-textioit-hdfs-500" created
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins4688283235771428210.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-textioit-hdfs-500
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins87464571919289008.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins4832975019125962512.sh
+ rm -rf .beam_env
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins6720836511915146889.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins8237513653598681044.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins716332382949683.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins8719587409380824653.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe 

Build failed in Jenkins: beam_PerformanceTests_XmlIOIT_HDFS #489

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam10 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins515286156721399073.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins1268229399290621868.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins1745172122392320273.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-xmlioit-hdfs-489
namespace "beam-performancetests-xmlioit-hdfs-489" created
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3843696738173512234.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-xmlioit-hdfs-489
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins2934916718808977001.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5346670940970426186.sh
+ rm -rf .beam_env
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3691206524504606188.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5859458165180357146.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins8950429006083433852.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3272729106275940420.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins879925326928

Build failed in Jenkins: beam_PerformanceTests_Compressed_TextIOIT_HDFS #490

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3539345491021498232.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7332140679707767447.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5237751151409750498.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-compressed-textioit-hdfs-490
namespace "beam-performancetests-compressed-textioit-hdfs-490" created
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins285647334703188770.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-compressed-textioit-hdfs-490
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7521618206738389411.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins9070722660594246530.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins594796153953716306.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins4089160093914930387.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins1377262095950994752.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Compressed_TextIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins2572188293259676732.sh
+ .beam_env/bin/

Build failed in Jenkins: beam_PerformanceTests_TFRecordIOIT #813

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins63334886438081341.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins3091900622950316868.sh
+ rm -rf .beam_env
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins3883249362530440042.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins1818752095345368157.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins6455615205446373512.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins7512727844650706579.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins8309866372636903617.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins9211654196471224361.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins475978130086332201.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.

Build failed in Jenkins: beam_PerformanceTests_TextIOIT #840

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam13 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins9005641272909315580.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins2075923558206330461.sh
+ rm -rf .beam_env
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins704200335023109.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins3577317678914270162.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins1181008640694015082.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins5508572205980109209.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins5269954315242232126.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins8624750945611832997.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins3714285359735661102.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in 
/home/jenkins/.local/lib/python2.7/site-pack

Build failed in Jenkins: beam_PerformanceTests_ParquetIOIT_HDFS #207

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam13 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins2899794644076596798.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins6491016650429032477.sh
+ cp /home/jenkins/.kube/config 

[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3074019208576167790.sh
+ kubectl 
--kubeconfig=
 create namespace beam-performancetests-parquetioit-hdfs-207
namespace "beam-performancetests-parquetioit-hdfs-207" created
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins3029353347243457680.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=beam-performancetests-parquetioit-hdfs-207
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins6170866838054443253.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins4058501399005310740.sh
+ rm -rf .beam_env
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5957120140553519021.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5566555065534541626.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins2565266911332028471.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_ParquetIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins143541014076919708.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/

Build failed in Jenkins: beam_PreCommit_Java_Cron #173

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
[...truncated 17.16 MB...]

Aug 01, 2018 12:21:26 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..4=[{inputs}], proj#0..1=[{exprs}])
  BeamJoinRel(condition=[AND(=($2, $4), >=($1, $3))], joinType=[inner])
BeamCalcRel(expr#0..2=[{inputs}], auction=[$t0], num=[$t2], 
starttime=[$t1])
  BeamAggregationRel(group=[{0, 1}], num=[COUNT()])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[5000], expr#6=[1], 
expr#7=[HOP($t3, $t5, $t6)], auction=[$t0], $f1=[$t7])
  BeamIOSourceRel(table=[[beam, Bid]])
BeamCalcRel(expr#0..1=[{inputs}], maxnum=[$t1], starttime=[$t0])
  BeamAggregationRel(group=[{1}], maxnum=[MAX($0)])
BeamCalcRel(expr#0..2=[{inputs}], num=[$t2], starttime=[$t1])
  BeamAggregationRel(group=[{0, 1}], num=[COUNT()])
BeamCalcRel(expr#0..4=[{inputs}], expr#5=[5000], 
expr#6=[1], expr#7=[HOP($t3, $t5, $t6)], auction=[$t0], $f1=[$t7])
  BeamIOSourceRel(table=[[beam, Bid]])


org.apache.beam.sdk.nexmark.queries.sql.SqlQuery3Test > 
testJoinsPeopleWithAuctions STANDARD_ERROR
Aug 01, 2018 12:21:27 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `P`.`name`, `P`.`city`, `P`.`state`, `A`.`id`
FROM `beam`.`Auction` AS `A`
INNER JOIN `beam`.`Person` AS `P` ON `A`.`seller` = `P`.`id`
WHERE `A`.`category` = 10 AND (`P`.`state` = 'OR' OR `P`.`state` = 'ID' OR 
`P`.`state` = 'CA')
Aug 01, 2018 12:21:27 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(name=[$11], city=[$14], state=[$15], id=[$0])
  LogicalFilter(condition=[AND(=($8, 10), OR(=($15, 'OR'), =($15, 'ID'), 
=($15, 'CA')))])
LogicalJoin(condition=[=($7, $10)], joinType=[inner])
  BeamIOSourceRel(table=[[beam, Auction]])
  BeamIOSourceRel(table=[[beam, Person]])

Aug 01, 2018 12:21:27 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..17=[{inputs}], name=[$t11], city=[$t14], state=[$t15], 
id=[$t0])
  BeamJoinRel(condition=[=($7, $10)], joinType=[inner])
BeamCalcRel(expr#0..9=[{inputs}], expr#10=[10], expr#11=[=($t8, $t10)], 
proj#0..9=[{exprs}], $condition=[$t11])
  BeamIOSourceRel(table=[[beam, Auction]])
BeamCalcRel(expr#0..7=[{inputs}], expr#8=['OR'], expr#9=[=($t5, $t8)], 
expr#10=['ID'], expr#11=[=($t5, $t10)], expr#12=['CA'], expr#13=[=($t5, $t12)], 
expr#14=[OR($t9, $t11, $t13)], proj#0..7=[{exprs}], $condition=[$t14])
  BeamIOSourceRel(table=[[beam, Person]])


org.apache.beam.sdk.nexmark.queries.sql.SqlQuery7Test > testBids STANDARD_ERROR
Aug 01, 2018 12:21:27 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, `B`.`extra`
FROM (SELECT `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, 
`B`.`extra`, TUMBLE_START(`B`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B`
GROUP BY `B`.`auction`, `B`.`price`, `B`.`bidder`, `B`.`dateTime`, 
`B`.`extra`, TUMBLE(`B`.`dateTime`, INTERVAL '10' SECOND)) AS `B`
INNER JOIN (SELECT MAX(`B1`.`price`) AS `maxprice`, 
TUMBLE_START(`B1`.`dateTime`, INTERVAL '10' SECOND) AS `starttime`
FROM `beam`.`Bid` AS `B1`
GROUP BY TUMBLE(`B1`.`dateTime`, INTERVAL '10' SECOND)) AS `B1` ON 
`B`.`starttime` = `B1`.`starttime` AND `B`.`price` = `B1`.`maxprice`
Aug 01, 2018 12:21:27 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], 
extra=[$4])
  LogicalJoin(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
LogicalProject(auction=[$0], price=[$1], bidder=[$2], dateTime=[$3], 
extra=[$4], starttime=[$5])
  LogicalAggregate(group=[{0, 1, 2, 3, 4, 5}])
LogicalProject(auction=[$0], price=[$2], bidder=[$1], 
dateTime=[$3], extra=[$4], $f5=[TUMBLE($3, 1)])
  BeamIOSourceRel(table=[[beam, Bid]])
LogicalProject(maxprice=[$1], starttime=[$0])
  LogicalAggregate(group=[{0}], maxprice=[MAX($1)])
LogicalProject($f0=[TUMBLE($3, 1)], price=[$2])
  BeamIOSourceRel(table=[[beam, Bid]])

Aug 01, 2018 12:21:27 PM 
org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..7=[{inputs}], proj#0..4=[{exprs}])
  BeamJoinRel(condition=[AND(=($5, $7), =($1, $6))], joinType=[inner])
BeamCal

Build failed in Jenkins: beam_PerformanceTests_ParquetIOIT #313

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins8779854765405935983.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins8854628596004831729.sh
+ rm -rf .beam_env
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins4150374710800053931.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins3398717206584055072.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins6796643233964836722.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins4801921422038179439.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins5054489901736011366.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins3794928425643452135.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_ParquetIOIT] $ /bin/bash -xe 
/tmp/jenkins4863680654604114284.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 i

Build failed in Jenkins: beam_PerformanceTests_Python #1262

[Apache Jenkins Server]

See 


--
[...truncated 2.19 KB...]
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins134203476966691273.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins266418180377408786.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins2763990076871907715.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins3751194031371478445.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins3369103436413740422.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins2139078975145550914.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins8720383783791910367.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins4964247970169696296.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins5972620634519769960.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.0.0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(4.1.2)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(3.6.0)
Requirement already satisfied: pydot<1.3,>=1.2.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.2.4)
Requirement already satisfied: pytz<=2018.4,>=2018.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2018.4)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in 
/usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (3.12)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.6.8)
Requirement already satisfied: six<1.12,>=1.9 in 
/usr/local/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.11.0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(3.6.4)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(3.2.0)
Req

Build failed in Jenkins: beam_PerformanceTests_Python #1263

[Apache Jenkins Server]

See 


Changes:

[rober] Avoid redundant computation when invoking ProcessElements in ParDo

[github] Update fn_test.go

[rober] De-dup pre & post invoke code in pardo.go

[timrobertson100] [BEAM-2661] Adds KuduIO

[github] Properly revert log line changes in fn_test.go

[rober] Add comparitive benchmark between cached and single use invoker.

[github] [BEAM-4852] Only read symbol table when required.

[github] Update symbols.go

[timrobertson100] [BEAM-2661] KuduIO: Add missing licenses

[ehudm] Make with_attributes kwarg optional.

[ehudm] Add decode and encode steps to streaming_wordcount

[qinyeli] Interactive Beam -- display update

[ehudm] Fix PubSubMessageMatcher not acking messages.

[ebeach] [BEAM-4862] Fixes bug in Spanner's MutationGroupEncoder by converting

[herohde] Add utilities for manipulating proto pipelines

[herohde] [BEAM-4636] Refactor Go dataflow runner for more flexible uses

[herohde] [BEAM-4813] Switch to translation from model protos to Dataflow 
v1beta3

[herohde] Add more Go integration tests and fix CoGBK bugs

[herohde] [BEAM-4636] Ensure unique filename for compiled worker binaries

[herohde] Handle explicit flatten

[herohde] Fix Dataflow argument bug

[herohde] CR: tweak code and fix comments

[aaltay] [BEAM-4751] fix missing pylint3 check for io subpackage (#5916)

[pablo] Add GitScm poll trigger for post-commit tests.

[relax] Enable schemas for Dataflow runner.

[tweise] [BEAM-5023] Send HarnessId in DataClient (#6066)

[relax] Fix BigQuery job retry mechanism.

[relax] Add exponential backoff on retries.

[relax] Apply code-review comment.

[tweise] [BEAM-2930] Side inputs are not yet supported in streaming mode.

[github] Don't rely on order of elements in a PCollection after GBK in

[relax] Properly handle running jobs.

[devinduan] Spelling mistakes

[relax] Convert BeamSQL to use Schemas.

[relax] Deprecate getRowCoder.

[relax] Add setSchema to remaining Table objects.

[relax] Delete a bunch of code that is no longer used.

[relax] Move utilities into schemas.utils package.

[relax] Convert Nexmark to use schemas.

[relax] remove deprecated Schema.getRowCoder

[relax] Apply spotless.

[relax] Plumb schema through DoFn.

[relax] Add helper setRowSchema to eliminatre boilerplate.

[rober] Avoid overwritting user changes to Resolver

[rober] Clean up deferedResolver

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

[pablo] Fix scheduling for jobs

[relax] Add convenience methods for pojo and javabean schema registration.

[relax] Address code-review comments.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ff95a82e461bd8319d9733be60e75992ba90cd7c # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins8260919133135875607.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins3883146302874079411.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins8573027766605549178.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Python] $ /bin/bash -xe 
/tmp/jenkins7954528295858307459.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Python] $ /bin/bash -xe /tm

Build failed in Jenkins: beam_PerformanceTests_AvroIOIT #820

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam14 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins1182505645108649029.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins9066379088098190854.sh
+ rm -rf .beam_env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins7062844523460890920.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins6095827419184151701.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins9137550466197986182.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins2734985251588016667.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins6396531218427440367.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins5860141581118253282.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_AvroIOIT] $ /bin/bash -xe 
/tmp/jenkins4479468858360775879.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.11.3)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in 
/home/jenkins/.local/lib/python2.7/site-pack

Build failed in Jenkins: beam_PerformanceTests_Compressed_TextIOIT #807

[Apache Jenkins Server]

See 


Changes:

[thw] [BEAM-4842] Update Flink Runner to Flink 1.5.2

[migryz] Add retries for gpg key query logic.

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam6 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision f2ae73be3d3d1cb7c3d111505f832aed44bc92eb (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f2ae73be3d3d1cb7c3d111505f832aed44bc92eb
Commit message: "Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2"
 > git rev-list --no-walk ddef7de5f0f232d70e77482163852ea1a06e8220 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins8290586113272308950.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins2647748453311707359.sh
+ rm -rf .beam_env
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins8372623728682834139.sh
+ rm -rf .perfkit_env
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins5934518815139903381.sh
+ virtualenv .beam_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins3824802778120006915.sh
+ virtualenv .perfkit_env --system-site-packages
New python executable in 

Also creating executable in 

Installing setuptools, pkg_resources, pip, wheel...done.
Running virtualenv with interpreter /usr/bin/python2
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins5714034912546932134.sh
+ .beam_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.beam_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in ./.beam_env/lib/python2.7/site-packages 
(18.0)
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins6470755715985949039.sh
+ .perfkit_env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in 
./.perfkit_env/lib/python2.7/site-packages (40.0.0)
Requirement already up-to-date: pip in 
./.perfkit_env/lib/python2.7/site-packages (18.0)
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins1082445159535975033.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Compressed_TextIOIT] $ /bin/bash -xe 
/tmp/jenkins323339188087322100.sh
+ .beam_env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.8.2)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.7.0.dev0) (1.7)
Requirement already satisfied: dill<=0.2.8.2,>=0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(0.2.8.2)
Requirement already satisfied: grpcio<2,>=1.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(1.13.0)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.7.0.dev0) 
(2.1.0)
Requirement already satisfied: httplib2<=0.11.3,>=0.8 in 
/home/jenkins/

[jira] [Work logged] (BEAM-5056) [SQL] Nullability of aggregation expressions isn't inferred properly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5056?focusedWorklogId=129672&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129672
 ]

ASF GitHub Bot logged work on BEAM-5056:


Author: ASF GitHub Bot
Created on: 01/Aug/18 12:42
Start Date: 01/Aug/18 12:42
Worklog Time Spent: 10m 
  Work Description: holdensmagicalunicorn commented on issue #6118: 
[BEAM-5056] [SQL] Fix nullability in output schema
URL: https://github.com/apache/beam/pull/6118#issuecomment-409561208
 
 
   @kanterov, thanks! I am a bot who has found some folks who might be able to 
help with the review:@kennknowles, @XuMingmin and @akedin


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129672)
Time Spent: 20m  (was: 10m)

> [SQL] Nullability of aggregation expressions isn't inferred properly
> 
>
> Key: BEAM-5056
> URL: https://issues.apache.org/jira/browse/BEAM-5056
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Gleb Kanterov
>Assignee: Xu Mingmin
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Given schema and rows:
> {code:java}
> Schema schema =
> Schema.builder()
> .addNullableField("f_int1", Schema.FieldType.INT32)
> .addNullableField("f_int2", Schema.FieldType.INT32)
> .build();
> List rows =
> TestUtils.RowsBuilder.of(schema)
> .addRows(null, null)
> .getRows();
> {code}
> Following query fails:
> {code:sql}
> SELECT AVG(f_int1) FROM PCOLLECTION GROUP BY f_int2
> {code}
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Field EXPR$0 is not 
> nullable{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5056) [SQL] Nullability of aggregation expressions isn't inferred properly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5056?focusedWorklogId=129671&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129671
 ]

ASF GitHub Bot logged work on BEAM-5056:


Author: ASF GitHub Bot
Created on: 01/Aug/18 12:42
Start Date: 01/Aug/18 12:42
Worklog Time Spent: 10m 
  Work Description: kanterov opened a new pull request #6118: [BEAM-5056] 
[SQL] Fix nullability in output schema
URL: https://github.com/apache/beam/pull/6118
 
 
   Before we didn't properly propagate nullability from Calcite to
   `Schema.Field`.
   
   In the case of "group by", aggregate expressions were always non-nullable.
   It isn't true if the bucketing key and aggregated expression are nullable.
   
   ```
   > SELECT SUM(EXPR$0), EXPR$0 FROM UNNEST (ARRAY [1, NULL]) GROUP BY EXPR$0;
   +++
   |   EXPR$0   |   EXPR$0   |
   +++
   | null   | null   |
   | 1  | 1  |
   +++
   ```
   
   In the case of "join", every field was nullable. It true for outer joins,
   but isn't precise for left, right or inner joins.
   
   The code is based on https://github.com/apache/beam/pull/6108. One 
suggestion would be to check output schema in tests, along with output.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129711&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129711
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206892840
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsCheckpointMark.java
 ##
 @@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.avro.reflect.AvroIgnore;
+import org.apache.beam.sdk.coders.AvroCoder;
+import org.apache.beam.sdk.coders.DefaultCoder;
+import org.apache.beam.sdk.io.UnboundedSource;
+
+@DefaultCoder(AvroCoder.class)
 
 Review comment:
   Maybe we can do this class just serializable and use the serializable coder 
in the getCheckpointMarkCoder method in the IO just to avoid Avro altogether.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129711)
Time Spent: 4h 20m  (was: 4h 10m)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 4h 20m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129715&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129715
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206900452
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
+  }
+
+  @Override
+  public Instant getCurrentTimestamp() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return Instant.parse(current.getAttributes().get("Timestamp"));
+  }
+
+  @Override
+  public Message getCurrent() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return current;
+  }
+
+  @Override
+  public CheckpointMark getCheckpointMark() {
+List snapshotReceiptHandlesToDelete = 
Lists.newArrayList(receiptHandlesToDelete);
+return new SqsCheckpointMark(this, snapshotReceiptHandlesToDelete);
+  }
+
+  @Override
+  public SqsUnboundedSource getCurrentSource() {
+return source;
+  }
+
+  @Override
+  public boolean start() {
+return advance();
+  }
+
+  @Override
+  public boolean advance() {
+if (messagesNotYetRead.isEmpty()) {
+  pull();
+}
+
+current = messagesNotYetRead.poll();
+if (current == null) {
+  return false;
+}
+
+receiptHandlesToDelete.add(current.getReceiptHandle());
+return true;
+  }
+
+  @Override
+  public void close() {}
+
+  void delete(String receiptHandle) {
+sqs.deleteMessage(source.getRead().queueUrl(), receiptHandle);
 
 Review comment:
   Thinking if we can have this method receiving a List of handles and then 
using `deleteMessageBatch`  to delete them (argh hope AWS does not limit by 10 
this but anyway will be better). Again not a bl

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129714
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206902198
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
+  }
+
+  @Override
+  public Instant getCurrentTimestamp() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return Instant.parse(current.getAttributes().get("Timestamp"));
+  }
+
+  @Override
+  public Message getCurrent() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return current;
+  }
+
+  @Override
+  public CheckpointMark getCheckpointMark() {
+List snapshotReceiptHandlesToDelete = 
Lists.newArrayList(receiptHandlesToDelete);
+return new SqsCheckpointMark(this, snapshotReceiptHandlesToDelete);
+  }
+
+  @Override
+  public SqsUnboundedSource getCurrentSource() {
+return source;
+  }
+
+  @Override
+  public boolean start() {
+return advance();
+  }
+
+  @Override
+  public boolean advance() {
+if (messagesNotYetRead.isEmpty()) {
+  pull();
+}
+
+current = messagesNotYetRead.poll();
+if (current == null) {
+  return false;
+}
+
+receiptHandlesToDelete.add(current.getReceiptHandle());
+return true;
+  }
+
+  @Override
+  public void close() {}
+
+  void delete(String receiptHandle) {
+sqs.deleteMessage(source.getRead().queueUrl(), receiptHandle);
 
 Review comment:
   Trying to understand also if the differences of Standard vs FIFO queues can 
have a consequence for checkpointing.


This is an automated

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129710&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129710
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206895285
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
+  }
+
+  @Override
+  public Instant getCurrentTimestamp() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return Instant.parse(current.getAttributes().get("Timestamp"));
 
 Review comment:
   Is this the correct attribute or is it "SentTimestamp" ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129710)
Time Spent: 4h 10m  (was: 4h)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 4h 10m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129708&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129708
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206789041
 
 

 ##
 File path: sdks/java/io/amazon-web-services/build.gradle
 ##
 @@ -42,4 +45,12 @@ dependencies {
   shadowTest library.java.mockito_core
   shadowTest library.java.junit
   shadowTest library.java.slf4j_jdk14
+  shadowTest group: 'org.elasticmq', name: 'elasticmq-rest-sqs_2.12', version: 
'0.14.1'
+}
+
+test {
 
 Review comment:
   Is this really needed to test with the embedded instance? looks interesting 
for documentation purposes or if we add an Integration Test but if not needed 
maybe remove and we add it later no?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129708)
Time Spent: 3h 50m  (was: 3h 40m)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129712&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129712
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206894530
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
 
 Review comment:
   This ignores the original message timestamps so probably worth to use an 
approach similar to what [Jms does (saving the oldest 
timestamp](https://github.com/apache/beam/blob/f2ae73be3d3d1cb7c3d111505f832aed44bc92eb/sdks/java/io/jms/src/main/java/org/apache/beam/sdk/io/jms/JmsCheckpointMark.java).
   
   Notice however that if the queue is not standard (the FIFO case) [just 
updating the watermark to the message 
timestamp](https://github.com/apache/beam/blob/f2ae73be3d3d1cb7c3d111505f832aed44bc92eb/sdks/java/io/amqp/src/main/java/org/apache/beam/sdk/io/amqp/AmqpIO.java#L308-L309)
 could be enough.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129712)
Time Spent: 4.5h  (was: 4h 20m)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> Add an S

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129709&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129709
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206885925
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsIO.java
 ##
 @@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import static com.google.common.base.Preconditions.checkArgument;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.google.auto.value.AutoValue;
+import javax.annotation.Nullable;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.io.aws.options.AwsOptions;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.joda.time.Duration;
+
+/**
+ * An unbounded source for Amazon Simple Queue Service (SQS).
+ *
+ * Reading from an SQS queue
+ *
+ * The {@link SqsIO} {@link Read} returns an unbounded {@link PCollection} 
of {@link
+ * com.amazonaws.services.sqs.model.Message} containing the received messages.
+ *
+ * To configure an SQS source, you have to provide the queueUrl to connect 
to. The following
+ * example illustrates how to configure the source:
+ *
+ * {@code
+ * pipeline.apply(SqsIO.read().withQueueUrl(queueUrl))
+ * }
+ *
+ * Writing to an SQS queue
+ *
+ * To configure an SQS sink, you have to provide the queueUrl to connect 
to. The following
+ * example illustrates how to configure the sink:
+ *
+ * {@code
+ * pipeline
+ *   .apply(...) // returns PCollection
+ *   .apply(SqsIO.write().withQueueUrl(queueUrl))
+ * }
+ *
+ * Additional Configuration
+ *
+ * Additional configuration can be provided via {@link AwsOptions} from 
command line args or in
+ * code. For example, if you wanted to provide a secret access key via code:
+ *
+ * {@code
+ * PipelineOptions pipelineOptions = 
PipelineOptionsFactory.fromArgs(args).withValidation().create();
+ * AwsOptions awsOptions = pipelineOptions.as(AwsOptions.class);
+ * BasicAWSCredentials awsCreds = new BasicAWSCredentials("accesskey", 
"secretkey");
+ * awsOptions.setAwsCredentialsProvider(new 
AWSStaticCredentialsProvider(awsCreds));
+ * Pipeline pipeline = Pipeline.create(options);
+ * }
+ *
+ * For more information on the available options see {@link AwsOptions}.
+ */
+@Experimental(Experimental.Kind.SOURCE_SINK)
+public class SqsIO {
+
+  public static Read read() {
+return new 
AutoValue_SqsIO_Read.Builder().setMaxNumRecords(Long.MAX_VALUE).build();
+  }
+
+  public static Write write() {
+return new AutoValue_SqsIO_Write.Builder().build();
+  }
+
+  private SqsIO() {}
+
+  /**
+   * A {@link PTransform} to read/receive messages from SQS. See {@link SqsIO} 
for more information
+   * on usage and configuration.
+   */
+  @AutoValue
+  public abstract static class Read extends PTransform> {
+
+@Nullable
+abstract String queueUrl();
+
+abstract long maxNumRecords();
+
+@Nullable
+abstract Duration maxReadTime();
+
+abstract Builder toBuilder();
+
+@AutoValue.Builder
+abstract static class Builder {
+  abstract Builder setQueueUrl(String queueUrl);
+
+  abstract Builder setMaxNumRecords(long maxNumRecords);
+
+  abstract Builder setMaxReadTime(Duration maxReadTime);
+
+  abstract Read build();
+}
+
+/**
+ * Define the max number of records received by the {@link Read}. When the 
max number of records
+ * is lower than {@code Long.MAX_VALUE}, the

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129713&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129713
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 14:46
Start Date: 01/Aug/18 14:46
Worklog Time Spent: 10m 
  Work Description: iemejia commented on a change in pull request #6101: 
[BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206894930
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
 
 Review comment:
   I am still wondering if it is worth to have different strategies depending 
if it uses a Standard or FIFO queue. But it seems the standard case should 
cover the timestamp approximately well for the FIFO one. Any ideas? (of course 
this can be subject of other JIRA)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129713)
Time Spent: 4h 40m  (was: 4.5h)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5050) [SQL] NULLs are aggregated incorrectly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5050?focusedWorklogId=129731&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129731
 ]

ASF GitHub Bot logged work on BEAM-5050:


Author: ASF GitHub Bot
Created on: 01/Aug/18 16:18
Start Date: 01/Aug/18 16:18
Worklog Time Spent: 10m 
  Work Description: vectorijk commented on issue #6108: [BEAM-5050] [SQL] 
Fix aggregation of nulls
URL: https://github.com/apache/beam/pull/6108#issuecomment-409632446
 
 
   looks good to me


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129731)
Time Spent: 0.5h  (was: 20m)

> [SQL] NULLs are aggregated incorrectly
> --
>
> Key: BEAM-5050
> URL: https://issues.apache.org/jira/browse/BEAM-5050
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Anton Kedin
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> For example, COUNT(field) should not count records with NULL field. We also 
> should handle and test on other aggregation functions (like AVG, SUM, MIN, 
> MAX, VAR_POP, VAR_SAMP, etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5056) [SQL] Nullability of aggregation expressions isn't inferred properly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5056?focusedWorklogId=129735&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129735
 ]

ASF GitHub Bot logged work on BEAM-5056:


Author: ASF GitHub Bot
Created on: 01/Aug/18 16:24
Start Date: 01/Aug/18 16:24
Worklog Time Spent: 10m 
  Work Description: akedin commented on issue #6118: [BEAM-5056] [SQL] Fix 
nullability in output schema
URL: https://github.com/apache/beam/pull/6118#issuecomment-409634416
 
 
   + @apilloud @amaliujia 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129735)
Time Spent: 0.5h  (was: 20m)

> [SQL] Nullability of aggregation expressions isn't inferred properly
> 
>
> Key: BEAM-5056
> URL: https://issues.apache.org/jira/browse/BEAM-5056
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Gleb Kanterov
>Assignee: Xu Mingmin
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Given schema and rows:
> {code:java}
> Schema schema =
> Schema.builder()
> .addNullableField("f_int1", Schema.FieldType.INT32)
> .addNullableField("f_int2", Schema.FieldType.INT32)
> .build();
> List rows =
> TestUtils.RowsBuilder.of(schema)
> .addRows(null, null)
> .getRows();
> {code}
> Following query fails:
> {code:sql}
> SELECT AVG(f_int1) FROM PCOLLECTION GROUP BY f_int2
> {code}
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Field EXPR$0 is not 
> nullable{code}
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129737&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129737
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 16:37
Start Date: 01/Aug/18 16:37
Worklog Time Spent: 10m 
  Work Description: JohnRudolfLewis commented on a change in pull request 
#6101: [BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206949674
 
 

 ##
 File path: sdks/java/io/amazon-web-services/build.gradle
 ##
 @@ -42,4 +45,12 @@ dependencies {
   shadowTest library.java.mockito_core
   shadowTest library.java.junit
   shadowTest library.java.slf4j_jdk14
+  shadowTest group: 'org.elasticmq', name: 'elasticmq-rest-sqs_2.12', version: 
'0.14.1'
+}
+
+test {
 
 Review comment:
   The only tests currently in this PR depends on this in-memory SQS compatible 
queue. Mocking would take more effort in my opinion than it is worth in order 
to be a pure unit test as compared to using this.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129737)
Time Spent: 5h  (was: 4h 50m)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 5h
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4936) Beam Dependency Update Request: org.codehaus.groovy

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4936?focusedWorklogId=129748&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129748
 ]

ASF GitHub Bot logged work on BEAM-4936:


Author: ASF GitHub Bot
Created on: 01/Aug/18 16:59
Start Date: 01/Aug/18 16:59
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #6115: DO NOT MERGE, 
[BEAM-4936] update org.apache.httpcomponents
URL: https://github.com/apache/beam/pull/6115#issuecomment-409646394
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129748)
Time Spent: 20m  (was: 10m)

> Beam Dependency Update Request: org.codehaus.groovy
> ---
>
> Key: BEAM-4936
> URL: https://issues.apache.org/jira/browse/BEAM-4936
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 2018-07-25 20:26:44.528984
> Please review and upgrade the org.codehaus.groovy to the latest 
> version None 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129751&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129751
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:02
Start Date: 01/Aug/18 17:02
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4943: [BEAM-3906] Automate 
Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409647601
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129751)
Time Spent: 28h  (was: 27h 50m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 28h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129755&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129755
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:06
Start Date: 01/Aug/18 17:06
Worklog Time Spent: 10m 
  Work Description: JohnRudolfLewis commented on a change in pull request 
#6101: [BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206959322
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsCheckpointMark.java
 ##
 @@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.avro.reflect.AvroIgnore;
+import org.apache.beam.sdk.coders.AvroCoder;
+import org.apache.beam.sdk.coders.DefaultCoder;
+import org.apache.beam.sdk.io.UnboundedSource;
+
+@DefaultCoder(AvroCoder.class)
 
 Review comment:
   will do


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129755)
Time Spent: 5h 10m  (was: 5h)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 5h 10m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129757&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129757
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:07
Start Date: 01/Aug/18 17:07
Worklog Time Spent: 10m 
  Work Description: JohnRudolfLewis commented on a change in pull request 
#6101: [BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206959484
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsIO.java
 ##
 @@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import static com.google.common.base.Preconditions.checkArgument;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.google.auto.value.AutoValue;
+import javax.annotation.Nullable;
+import org.apache.beam.sdk.annotations.Experimental;
+import org.apache.beam.sdk.io.aws.options.AwsOptions;
+import org.apache.beam.sdk.transforms.DoFn;
+import org.apache.beam.sdk.transforms.PTransform;
+import org.apache.beam.sdk.transforms.ParDo;
+import org.apache.beam.sdk.values.PBegin;
+import org.apache.beam.sdk.values.PCollection;
+import org.apache.beam.sdk.values.PDone;
+import org.joda.time.Duration;
+
+/**
+ * An unbounded source for Amazon Simple Queue Service (SQS).
+ *
+ * Reading from an SQS queue
+ *
+ * The {@link SqsIO} {@link Read} returns an unbounded {@link PCollection} 
of {@link
+ * com.amazonaws.services.sqs.model.Message} containing the received messages.
+ *
+ * To configure an SQS source, you have to provide the queueUrl to connect 
to. The following
+ * example illustrates how to configure the source:
+ *
+ * {@code
+ * pipeline.apply(SqsIO.read().withQueueUrl(queueUrl))
+ * }
+ *
+ * Writing to an SQS queue
+ *
+ * To configure an SQS sink, you have to provide the queueUrl to connect 
to. The following
+ * example illustrates how to configure the sink:
+ *
+ * {@code
+ * pipeline
+ *   .apply(...) // returns PCollection
+ *   .apply(SqsIO.write().withQueueUrl(queueUrl))
+ * }
+ *
+ * Additional Configuration
+ *
+ * Additional configuration can be provided via {@link AwsOptions} from 
command line args or in
+ * code. For example, if you wanted to provide a secret access key via code:
+ *
+ * {@code
+ * PipelineOptions pipelineOptions = 
PipelineOptionsFactory.fromArgs(args).withValidation().create();
+ * AwsOptions awsOptions = pipelineOptions.as(AwsOptions.class);
+ * BasicAWSCredentials awsCreds = new BasicAWSCredentials("accesskey", 
"secretkey");
+ * awsOptions.setAwsCredentialsProvider(new 
AWSStaticCredentialsProvider(awsCreds));
+ * Pipeline pipeline = Pipeline.create(options);
+ * }
+ *
+ * For more information on the available options see {@link AwsOptions}.
+ */
+@Experimental(Experimental.Kind.SOURCE_SINK)
+public class SqsIO {
+
+  public static Read read() {
+return new 
AutoValue_SqsIO_Read.Builder().setMaxNumRecords(Long.MAX_VALUE).build();
+  }
+
+  public static Write write() {
+return new AutoValue_SqsIO_Write.Builder().build();
+  }
+
+  private SqsIO() {}
+
+  /**
+   * A {@link PTransform} to read/receive messages from SQS. See {@link SqsIO} 
for more information
+   * on usage and configuration.
+   */
+  @AutoValue
+  public abstract static class Read extends PTransform> {
+
+@Nullable
+abstract String queueUrl();
+
+abstract long maxNumRecords();
+
+@Nullable
+abstract Duration maxReadTime();
+
+abstract Builder toBuilder();
+
+@AutoValue.Builder
+abstract static class Builder {
+  abstract Builder setQueueUrl(String queueUrl);
+
+  abstract Builder setMaxNumRecords(long maxNumRecords);
+
+  abstract Builder setMaxReadTime(Duration maxReadTime);
+
+  abstract Read build();
+}
+
+/**
+ * Define the max number of records received by the {@link Read}. When the 
max number of records
+ * is lower than {@code Long.MAX_VAL

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129763&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129763
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:12
Start Date: 01/Aug/18 17:12
Worklog Time Spent: 10m 
  Work Description: JohnRudolfLewis commented on a change in pull request 
#6101: [BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206961020
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
 
 Review comment:
   Perhaps I misread the javadoc comments on the base class...
   
   "For a source that does not have natural timestamps, timestamps can be set 
to the time of reading, in which case the watermark is the current clock time."
   
   
https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/io/UnboundedSource.java#L217-L219
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129763)
Time Spent: 5.5h  (was: 5h 20m)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129756&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129756
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:06
Start Date: 01/Aug/18 17:06
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4943: [BEAM-3906] Automate 
Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409648800
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129756)
Time Spent: 28h 10m  (was: 28h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 28h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-4826) Flink runner sends bad flatten to SDK

[Henning Rohde (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Henning Rohde reassigned BEAM-4826:
---

Assignee: Ankur Goenka  (was: Aljoscha Krettek)

> Flink runner sends bad flatten to SDK
> -
>
> Key: BEAM-4826
> URL: https://issues.apache.org/jira/browse/BEAM-4826
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Henning Rohde
>Assignee: Ankur Goenka
>Priority: Major
>  Labels: portability
>
> For a Go flatten test w/ 3 input, the Flink runner splits this into 3 bundle 
> descriptors. But it sends the original 3-input flatten but w/ 1 actual input 
> present in each bundle descriptor. This is inconsistent and the SDK shouldn't 
> expect dangling PCollections. In contrast, Dataflow removes the flatten when 
> it does the same split.
> Snippet:
> register: <
>   process_bundle_descriptor: <
> id: "3"
> transforms: <
>   key: "e4"
>   value: <
> unique_name: "github.com/apache/beam/sdks/go/pkg/beam.createFn'1"
> spec: <
>   urn: "urn:beam:transform:pardo:v1"
>   payload: [...]
> >
> inputs: <
>   key: "i0"
>   value: "n3"
> >
> outputs: <
>   key: "i0"
>   value: "n4"
> >
>   >
> >
> transforms: <
>   key: "e7"
>   value: <
> unique_name: "Flatten"
> spec: <
>   urn: "beam:transform:flatten:v1"
> >
> inputs: <
>   key: "i0"
>   value: "n2"
> >
> inputs: <
>   key: "i1"
>   value: "n4" . // <--- only one present.
> >
> inputs: <
>   key: "i2"
>   value: "n6"
> >
> outputs: <
>   key: "i0"
>   value: "n7"
> >
>   >
> >
> [...]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129764&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129764
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:13
Start Date: 01/Aug/18 17:13
Worklog Time Spent: 10m 
  Work Description: JohnRudolfLewis commented on a change in pull request 
#6101: [BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206961417
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
+  }
+
+  @Override
+  public Instant getCurrentTimestamp() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return Instant.parse(current.getAttributes().get("Timestamp"));
 
 Review comment:
   It is the attribute that I am setting on line 142. Would you like me to 
extract a constant? I will.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129764)
Time Spent: 5h 40m  (was: 5.5h)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 5h 40m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to follow progress or help out: 
> [https://github.com/JohnRudolfLewis/beam/tree/Add-SqsIO]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] 01/01: Merge pull request #6108 : [BEAM-5050] [SQL] Fix aggregation of nulls

[mingmxu]

This is an automated email from the ASF dual-hosted git repository.

mingmxu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit 4310cbf78b62b2d4948978df95197f5347f08c02
Merge: f2ae73b 109f39e
Author: XuMingmin 
AuthorDate: Wed Aug 1 10:15:56 2018 -0700

Merge pull request #6108 : [BEAM-5050] [SQL] Fix aggregation of nulls

[BEAM-5050] [SQL] Fix aggregation of nulls

 .../impl/transform/BeamAggregationTransforms.java  |  48 ---
 .../extensions/sql/impl/utils/CalciteUtils.java|   4 +-
 .../sql/BeamSqlDslAggregationNullableTest.java | 158 +
 .../extensions/sql/impl/rel/BeamSortRelTest.java   |  11 +-
 .../sql/impl/utils/CalciteUtilsTest.java   | 127 +
 .../beam/sdk/extensions/sql/utils/RowAsserts.java  |  10 ++
 6 files changed, 335 insertions(+), 23 deletions(-)

[beam] branch master updated (f2ae73b -> 4310cbf)

[mingmxu]

This is an automated email from the ASF dual-hosted git repository.

mingmxu pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from f2ae73b  Merge pull request #6109: [BEAM-4842] Update Flink Runner to 
Flink 1.5.2
 add 109f39e  [BEAM-5050] [SQL] Fix aggregation of nulls
 new 4310cbf  Merge pull request #6108 : [BEAM-5050] [SQL] Fix aggregation 
of nulls

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 .../impl/transform/BeamAggregationTransforms.java  |  48 ---
 .../extensions/sql/impl/utils/CalciteUtils.java|   4 +-
 .../sql/BeamSqlDslAggregationNullableTest.java | 158 +
 .../extensions/sql/impl/rel/BeamSortRelTest.java   |  11 +-
 .../sql/impl/utils/CalciteUtilsTest.java   | 127 +
 .../beam/sdk/extensions/sql/utils/RowAsserts.java  |  10 ++
 6 files changed, 335 insertions(+), 23 deletions(-)
 create mode 100644 
sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/BeamSqlDslAggregationNullableTest.java
 create mode 100644 
sdks/java/extensions/sql/src/test/java/org/apache/beam/sdk/extensions/sql/impl/utils/CalciteUtilsTest.java

[jira] [Work logged] (BEAM-5050) [SQL] NULLs are aggregated incorrectly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5050?focusedWorklogId=129765&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129765
 ]

ASF GitHub Bot logged work on BEAM-5050:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:15
Start Date: 01/Aug/18 17:15
Worklog Time Spent: 10m 
  Work Description: XuMingmin commented on issue #6108: [BEAM-5050] [SQL] 
Fix aggregation of nulls
URL: https://github.com/apache/beam/pull/6108#issuecomment-409651369
 
 
   Thanks @kanterov , merging


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129765)
Time Spent: 40m  (was: 0.5h)

> [SQL] NULLs are aggregated incorrectly
> --
>
> Key: BEAM-5050
> URL: https://issues.apache.org/jira/browse/BEAM-5050
> Project: Beam
>  Issue Type: Bug
>  Components: dsl-sql
>Reporter: Anton Kedin
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> For example, COUNT(field) should not count records with NULL field. We also 
> should handle and test on other aggregation functions (like AVG, SUM, MIN, 
> MAX, VAR_POP, VAR_SAMP, etc.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5050) [SQL] NULLs are aggregated incorrectly

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5050?focusedWorklogId=129766&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129766
 ]

ASF GitHub Bot logged work on BEAM-5050:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:15
Start Date: 01/Aug/18 17:15
Worklog Time Spent: 10m 
  Work Description: XuMingmin closed pull request #6108: [BEAM-5050] [SQL] 
Fix aggregation of nulls
URL: https://github.com/apache/beam/pull/6108
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamAggregationTransforms.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamAggregationTransforms.java
index 2d257ecb03b..a19d55e2fee 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamAggregationTransforms.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/transform/BeamAggregationTransforms.java
@@ -244,33 +244,43 @@ public AggregationAccumulator createAccumulator() {
 public AggregationAccumulator addInput(AggregationAccumulator accumulator, 
Row input) {
   AggregationAccumulator deltaAcc = new AggregationAccumulator();
   for (int idx = 0; idx < aggregators.size(); ++idx) {
+CombineFn aggregator = aggregators.get(idx);
+Object element = accumulator.accumulatorElements.get(idx);
+
 if (sourceFieldExps.get(idx) instanceof BeamSqlInputRefExpression) {
   BeamSqlInputRefExpression exp = (BeamSqlInputRefExpression) 
sourceFieldExps.get(idx);
-  deltaAcc.accumulatorElements.add(
-  aggregators
-  .get(idx)
-  .addInput(
-  accumulator.accumulatorElements.get(idx),
-  exp.evaluate(input, null, 
BeamSqlExpressionEnvironments.empty()).getValue()));
+  Object value =
+  exp.evaluate(input, null, 
BeamSqlExpressionEnvironments.empty()).getValue();
+
+  // every aggregator ignores null values, e.g., COUNT(NULL) is always 
zero
+  if (value != null) {
+Object delta = aggregator.addInput(element, value);
+deltaAcc.accumulatorElements.add(delta);
+  } else {
+deltaAcc.accumulatorElements.add(element);
+  }
 } else if (sourceFieldExps.get(idx) instanceof KV) {
-  /**
+  /*
* If source expression is type of KV pair, we bundle the value of 
two expressions into KV
* pair and pass it to aggregator's addInput method.
*/
   KV exp =
   (KV) 
sourceFieldExps.get(idx);
-  deltaAcc.accumulatorElements.add(
-  aggregators
-  .get(idx)
-  .addInput(
-  accumulator.accumulatorElements.get(idx),
-  KV.of(
-  exp.getKey()
-  .evaluate(input, null, 
BeamSqlExpressionEnvironments.empty())
-  .getValue(),
-  exp.getValue()
-  .evaluate(input, null, 
BeamSqlExpressionEnvironments.empty())
-  .getValue(;
+
+  Object key =
+  exp.getKey().evaluate(input, null, 
BeamSqlExpressionEnvironments.empty()).getValue();
+
+  Object value =
+  exp.getValue()
+  .evaluate(input, null, BeamSqlExpressionEnvironments.empty())
+  .getValue();
+
+  // ignore aggregator if either key or value is null, e.g., 
COVAR_SAMP(x, NULL) is null
+  if (key != null && value != null) {
+deltaAcc.accumulatorElements.add(aggregator.addInput(element, 
KV.of(key, value)));
+  } else {
+deltaAcc.accumulatorElements.add(element);
+  }
 }
   }
   return deltaAcc;
diff --git 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/utils/CalciteUtils.java
 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/utils/CalciteUtils.java
index 19044459f2a..e307b768e08 100644
--- 
a/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/utils/CalciteUtils.java
+++ 
b/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/utils/CalciteUtils.java
@@ -190,6 +190,8 @@ private static RelDataType toRelDataType(
   private static RelDataType toRelDataType(
   RelDataTypeFactory dataTypeFactory, Schema schema, int fieldIndex) {
 

[jira] [Work logged] (BEAM-4828) Add Amazon SqsIO

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4828?focusedWorklogId=129768&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129768
 ]

ASF GitHub Bot logged work on BEAM-4828:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:18
Start Date: 01/Aug/18 17:18
Worklog Time Spent: 10m 
  Work Description: JohnRudolfLewis commented on a change in pull request 
#6101: [BEAM-4828] Add Amazon SqsIO
URL: https://github.com/apache/beam/pull/6101#discussion_r206963089
 
 

 ##
 File path: 
sdks/java/io/amazon-web-services/src/main/java/org/apache/beam/sdk/io/aws/sqs/SqsUnboundedReader.java
 ##
 @@ -0,0 +1,154 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.aws.sqs;
+
+import com.amazonaws.services.sqs.AmazonSQS;
+import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
+import com.amazonaws.services.sqs.model.Message;
+import com.amazonaws.services.sqs.model.ReceiveMessageRequest;
+import com.amazonaws.services.sqs.model.ReceiveMessageResult;
+import com.google.common.collect.Lists;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayDeque;
+import java.util.HashSet;
+import java.util.List;
+import java.util.NoSuchElementException;
+import java.util.Queue;
+import java.util.Set;
+import org.apache.beam.sdk.io.UnboundedSource;
+import org.apache.beam.sdk.io.UnboundedSource.CheckpointMark;
+import org.joda.time.Instant;
+
+class SqsUnboundedReader extends UnboundedSource.UnboundedReader {
+
+  public static final int MAX_NUMBER_OF_MESSAGES = 10;
+  private final SqsUnboundedSource source;
+  private final AmazonSQS sqs;
+
+  private Message current;
+  private final Queue messagesNotYetRead;
+  private Set receiptHandlesToDelete;
+
+  public SqsUnboundedReader(SqsUnboundedSource source, SqsCheckpointMark 
sqsCheckpointMark) {
+this.source = source;
+this.current = null;
+
+this.messagesNotYetRead = new ArrayDeque<>();
+receiptHandlesToDelete = new HashSet<>();
+
+final SqsConfiguration sqsConfiguration = source.getSqsConfiguration();
+sqs =
+AmazonSQSClientBuilder.standard()
+.withClientConfiguration(sqsConfiguration.getClientConfiguration())
+.withCredentials(sqsConfiguration.getAwsCredentialsProvider())
+.withRegion(sqsConfiguration.getAwsRegion())
+.build();
+
+if (sqsCheckpointMark != null) {
+  if (sqsCheckpointMark.getReceiptHandlesToDelete() != null) {
+
receiptHandlesToDelete.addAll(sqsCheckpointMark.getReceiptHandlesToDelete());
+  }
+}
+  }
+
+  @Override
+  public Instant getWatermark() {
+return Instant.now();
+  }
+
+  @Override
+  public Instant getCurrentTimestamp() throws NoSuchElementException {
+if (current == null) {
+  throw new NoSuchElementException();
+}
+return Instant.parse(current.getAttributes().get("Timestamp"));
 
 Review comment:
   No... you were actually referring to SentTimestamp... an attribute added by 
SQS I missed that in the SQS docs initially, I did not see any 
timestamp property so assumed that they did not include this concept. Silly 
that they would hide it in an attribute. I need to refactor based on this new 
knowledge.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129768)
Time Spent: 5h 50m  (was: 5h 40m)

> Add Amazon SqsIO
> 
>
> Key: BEAM-4828
> URL: https://issues.apache.org/jira/browse/BEAM-4828
> Project: Beam
>  Issue Type: New Feature
>  Components: io-java-aws
>Reporter: John Rudolf Lewis
>Assignee: John Rudolf Lewis
>Priority: Major
>  Time Spent: 5h 50m
>  Remaining Estimate: 0h
>
> Add an SQS source
>  
> For people who would like to

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129770&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129770
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:20
Start Date: 01/Aug/18 17:20
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409648800
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129770)
Time Spent: 28.5h  (was: 28h 20m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 28.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129769&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129769
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:20
Start Date: 01/Aug/18 17:20
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4943: [BEAM-3906] Automate 
Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409652916
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129769)
Time Spent: 28h 20m  (was: 28h 10m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 28h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129779&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129779
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:26
Start Date: 01/Aug/18 17:26
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409647601
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129779)
Time Spent: 29h  (was: 28h 50m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 29h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129777&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129777
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:26
Start Date: 01/Aug/18 17:26
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4943: [BEAM-3906] Automate 
Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409655073
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129777)
Time Spent: 28h 40m  (was: 28.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 28h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129778&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129778
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:26
Start Date: 01/Aug/18 17:26
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409652916
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129778)
Time Spent: 28h 50m  (was: 28h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 28h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4936) Beam Dependency Update Request: org.codehaus.groovy

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4936?focusedWorklogId=129781&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129781
 ]

ASF GitHub Bot logged work on BEAM-4936:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:31
Start Date: 01/Aug/18 17:31
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #6115: DO NOT 
MERGE, [BEAM-4936] update org.apache.httpcomponents
URL: https://github.com/apache/beam/pull/6115#issuecomment-409441890
 
 
   Run Java PreCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129781)
Time Spent: 0.5h  (was: 20m)

> Beam Dependency Update Request: org.codehaus.groovy
> ---
>
> Key: BEAM-4936
> URL: https://issues.apache.org/jira/browse/BEAM-4936
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> 2018-07-25 20:26:44.528984
> Please review and upgrade the org.codehaus.groovy to the latest 
> version None 
>  
> cc: 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129787&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129787
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-388994453
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129787)
Time Spent: 29h 50m  (was: 29h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 29h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129791&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129791
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389010582
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129791)
Time Spent: 30.5h  (was: 30h 20m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 30.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129783&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129783
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou commented on issue #4943: [BEAM-3906] Automate 
Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409658054
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129783)
Time Spent: 29h 10m  (was: 29h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 29h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129786&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129786
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-384777276
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129786)
Time Spent: 29h 40m  (was: 29.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 29h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129793&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129793
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389013696
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129793)
Time Spent: 30h 50m  (was: 30h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 30h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129792&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129792
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389012910
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129792)
Time Spent: 30h 40m  (was: 30.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 30h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129789&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129789
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-388998501
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129789)
Time Spent: 30h 10m  (was: 30h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 30h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129790&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129790
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389009670
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129790)
Time Spent: 30h 20m  (was: 30h 10m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 30h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129784
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-409655073
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129784)
Time Spent: 29h 20m  (was: 29h 10m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 29h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129785&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129785
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:35
Start Date: 01/Aug/18 17:35
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-383226575
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129785)
Time Spent: 29.5h  (was: 29h 20m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 29.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129812&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129812
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-398566171
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129812)
Time Spent: 34h  (was: 33h 50m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 34h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129808&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129808
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-393624655
 
 
   Blocked until 2.5.0 release


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129808)
Time Spent: 33h 20m  (was: 33h 10m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 33h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129803&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129803
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389035498
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129803)
Time Spent: 32.5h  (was: 32h 20m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 32.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129795&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129795
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389018952
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129795)
Time Spent: 31h 10m  (was: 31h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 31h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129799&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129799
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389025496
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129799)
Time Spent: 31h 50m  (was: 31h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 31h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129796&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129796
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389019584
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129796)
Time Spent: 31h 20m  (was: 31h 10m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 31h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129801&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129801
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389027362
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129801)
Time Spent: 32h 10m  (was: 32h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 32h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129794&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129794
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389014309
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129794)
Time Spent: 31h  (was: 30h 50m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 31h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129811&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129811
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-398564924
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129811)
Time Spent: 33h 50m  (was: 33h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 33h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129810&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129810
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-397904986
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129810)
Time Spent: 33h 40m  (was: 33.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 33h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129798&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129798
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389022141
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129798)
Time Spent: 31h 40m  (was: 31.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 31h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129806&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129806
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389226945
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129806)
Time Spent: 33h  (was: 32h 50m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 33h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129805&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129805
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389036123
 
 
   Run Python Dataflow ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129805)
Time Spent: 32h 50m  (was: 32h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 32h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129813&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129813
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-398574865
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129813)
Time Spent: 34h 10m  (was: 34h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 34h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129800&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129800
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389026314
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129800)
Time Spent: 32h  (was: 31h 50m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 32h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129817&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129817
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-400137522
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129817)
Time Spent: 34h 50m  (was: 34h 40m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 34h 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129816&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129816
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-400136651
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129816)
Time Spent: 34h 40m  (was: 34.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 34h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129807&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129807
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389228769
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129807)
Time Spent: 33h 10m  (was: 33h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 33h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129809&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129809
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-397902561
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129809)
Time Spent: 33.5h  (was: 33h 20m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 33.5h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129802&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129802
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389027957
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129802)
Time Spent: 32h 20m  (was: 32h 10m)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 32h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-3906) Get Python Wheel Validation Automated

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-3906?focusedWorklogId=129804&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-129804
 ]

ASF GitHub Bot logged work on BEAM-3906:


Author: ASF GitHub Bot
Created on: 01/Aug/18 17:36
Start Date: 01/Aug/18 17:36
Worklog Time Spent: 10m 
  Work Description: yifanzou removed a comment on issue #4943: [BEAM-3906] 
Automate Validation Aganist Python Wheel
URL: https://github.com/apache/beam/pull/4943#issuecomment-389036091
 
 
   Run Python ReleaseCandidate


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 129804)
Time Spent: 32h 40m  (was: 32.5h)

> Get Python Wheel Validation Automated
> -
>
> Key: BEAM-3906
> URL: https://issues.apache.org/jira/browse/BEAM-3906
> Project: Beam
>  Issue Type: Sub-task
>  Components: examples-python, testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
>  Time Spent: 32h 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)