[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=153002=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153002
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 10/Oct/18 00:46
Start Date: 10/Oct/18 00:46
Worklog Time Spent: 10m
Work Description: jasonkuster closed pull request #6559: [BEAM-5630] add
more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/java/io/google-cloud-platform/build.gradle
b/sdks/java/io/google-cloud-platform/build.gradle
index 7ee0d66c41a..4e8c5fb8cce 100644
--- a/sdks/java/io/google-cloud-platform/build.gradle
+++ b/sdks/java/io/google-cloud-platform/build.gradle
@@ -97,6 +97,7 @@ task integrationTest(type: Test) {
outputs.upToDateWhen { false }
include '**/*IT.class'
+ exclude '**/BigQueryIOReadIT.class'
exclude '**/BigQueryToTableIT.class'
maxParallelForks 4
classpath = sourceSets.test.runtimeClasspath
diff --git
a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java
b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java
index 61141efd8a1..637a03b9944 100644
---
a/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java
+++
b/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java
@@ -18,11 +18,13 @@
package org.apache.beam.sdk.io.gcp.bigquery;
+import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
import java.util.Map;
import org.apache.beam.sdk.Pipeline;
import org.apache.beam.sdk.extensions.gcp.options.GcpOptions;
import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.ExperimentalOptions;
import org.apache.beam.sdk.options.PipelineOptionsFactory;
import org.apache.beam.sdk.options.Validation;
import org.apache.beam.sdk.testing.PAssert;
@@ -45,10 +47,15 @@
private String project;
private static final String datasetId = "big_query_import_export";
private static final String tablePrefix = "export_";
- private static final Map numOfRecords =
ImmutableMap.of("empty", 0, "1K", 1000);
+ private static final Map numOfRecords =
+ ImmutableMap.of(
+ "empty", 0L,
+ "1M", 10592L,
+ "1G", 0839L,
+ "1T", 0839000L);
/** Customized PipelineOption for BigQueryIORead Pipeline. */
- public interface BigQueryIOReadOptions extends TestPipelineOptions {
+ public interface BigQueryIOReadOptions extends TestPipelineOptions,
ExperimentalOptions {
@Description("The table to be read")
@Validation.Required
String getInputTable();
@@ -62,13 +69,17 @@
void setNumRecords(long numRecords);
}
- private void setupTestEnvironment(String recordSize) {
+ private void setupTestEnvironment(String recordSize, boolean
enableCustomBigquery) {
PipelineOptionsFactory.register(BigQueryIOReadOptions.class);
options =
TestPipeline.testingPipelineOptions().as(BigQueryIOReadOptions.class);
options.setNumRecords(numOfRecords.get(recordSize));
options.setTempLocation(options.getTempRoot() + "/temp-it/");
project =
TestPipeline.testingPipelineOptions().as(GcpOptions.class).getProject();
options.setInputTable(project + ":" + datasetId + "." + tablePrefix +
recordSize);
+if (enableCustomBigquery) {
+ options.setExperiments(
+ ImmutableList.of("enable_custom_bigquery_sink",
"enable_custom_bigquery_source"));
+}
}
private void runBigQueryIOReadPipeline() {
@@ -83,13 +94,37 @@ private void runBigQueryIOReadPipeline() {
@Test
public void testBigQueryReadEmpty() throws Exception {
-setupTestEnvironment("empty");
+setupTestEnvironment("empty", false);
runBigQueryIOReadPipeline();
}
@Test
- public void testBigQueryRead1K() throws Exception {
-setupTestEnvironment("1K");
+ public void testBigQueryRead1M() throws Exception {
+setupTestEnvironment("1M", false);
+runBigQueryIOReadPipeline();
+ }
+
+ @Test
+ public void testBigQueryRead1G() throws Exception {
+setupTestEnvironment("1G", false);
+runBigQueryIOReadPipeline();
+ }
+
+ @Test
+ public void testBigQueryRead1T() throws Exception {
+setupTestEnvironment("1T", false);
+runBigQueryIOReadPipeline();
+ }
+
+ @Test
+ public void testBigQueryReadEmptyCustom() throws Exception {
+setupTestEnvironment("empty", true);
+runBigQueryIOReadPipeline();
+ }
+
+ @Test
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=153001=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153001 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 10/Oct/18 00:45 Start Date: 10/Oct/18 00:45 Worklog Time Spent: 10m Work Description: jasonkuster commented on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-428400120 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 153001) Time Spent: 3h 40m (was: 3.5h) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 3h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=152842=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152842
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 09/Oct/18 18:23
Start Date: 09/Oct/18 18:23
Worklog Time Spent: 10m
Work Description: yifanzou commented on a change in pull request #6559:
[BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r223812009
##
File path: sdks/java/io/google-cloud-platform/build.gradle
##
@@ -97,6 +97,7 @@ task integrationTest(type: Test) {
outputs.upToDateWhen { false }
include '**/*IT.class'
+ exclude '**/BigQueryIOReadIT.class'
Review comment:
To be more specific, this test will be skipped when jobs running against the
DirectRunner since limited JVM heap. The entire class will still run against
the DataflowRunner in the job
[:beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest](https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/build.gradle#L152).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152842)
Time Spent: 3.5h (was: 3h 20m)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=152841=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152841
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 09/Oct/18 18:23
Start Date: 09/Oct/18 18:23
Worklog Time Spent: 10m
Work Description: yifanzou commented on a change in pull request #6559:
[BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r223812009
##
File path: sdks/java/io/google-cloud-platform/build.gradle
##
@@ -97,6 +97,7 @@ task integrationTest(type: Test) {
outputs.upToDateWhen { false }
include '**/*IT.class'
+ exclude '**/BigQueryIOReadIT.class'
Review comment:
To be more specific, this test will be skip when jobs running against the
DirectRunner since limited JVM heap. The entire class will still run against
the DataflowRunner in the job
[:beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest](https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/build.gradle#L152).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152841)
Time Spent: 3h 20m (was: 3h 10m)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=152830=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152830
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 09/Oct/18 18:07
Start Date: 09/Oct/18 18:07
Worklog Time Spent: 10m
Work Description: yifanzou commented on a change in pull request #6559:
[BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r223806946
##
File path: sdks/java/io/google-cloud-platform/build.gradle
##
@@ -97,6 +97,7 @@ task integrationTest(type: Test) {
outputs.upToDateWhen { false }
include '**/*IT.class'
+ exclude '**/BigQueryIOReadIT.class'
Review comment:
https://discuss.gradle.org/t/how-to-exclude-a-single-test-method/20860
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152830)
Time Spent: 3h 10m (was: 3h)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=152829=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152829
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 09/Oct/18 18:06
Start Date: 09/Oct/18 18:06
Worklog Time Spent: 10m
Work Description: yifanzou commented on a change in pull request #6559:
[BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r223806777
##
File path: sdks/java/io/google-cloud-platform/build.gradle
##
@@ -97,6 +97,7 @@ task integrationTest(type: Test) {
outputs.upToDateWhen { false }
include '**/*IT.class'
+ exclude '**/BigQueryIOReadIT.class'
Review comment:
Unfortunately no. The 'exclude' here only works on file pattern so that we
could only exclude the entire class. I found an incubating Gradle
[TestFilter](https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/testing/TestFilter.html)
that is able to execute specific test methods. But it doesn't work with the
'exclude' to skip test methods.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152829)
Time Spent: 3h (was: 2h 50m)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=152489=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152489
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 09/Oct/18 01:55
Start Date: 09/Oct/18 01:55
Worklog Time Spent: 10m
Work Description: jasonkuster commented on a change in pull request
#6559: [BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r223541717
##
File path: sdks/java/io/google-cloud-platform/build.gradle
##
@@ -97,6 +97,7 @@ task integrationTest(type: Test) {
outputs.upToDateWhen { false }
include '**/*IT.class'
+ exclude '**/BigQueryIOReadIT.class'
Review comment:
AIUI we don't want to exclude the entire test -- can we exclude only part of
it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152489)
Time Spent: 2h 50m (was: 2h 40m)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151446=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151446
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 05/Oct/18 00:20
Start Date: 05/Oct/18 00:20
Worklog Time Spent: 10m
Work Description: yifanzou commented on a change in pull request #6559:
[BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r222863957
##
File path:
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java
##
@@ -83,13 +94,31 @@ private void runBigQueryIOReadPipeline() {
@Test
public void testBigQueryReadEmpty() throws Exception {
-setupTestEnvironment("empty");
+setupTestEnvironment("empty", false);
runBigQueryIOReadPipeline();
}
@Test
- public void testBigQueryRead1K() throws Exception {
-setupTestEnvironment("1K");
+ public void testBigQueryRead1G() throws Exception {
Review comment:
done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151446)
Time Spent: 2.5h (was: 2h 20m)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151447=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151447 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 05/Oct/18 00:20 Start Date: 05/Oct/18 00:20 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-427209430 @jasonkuster PTAL, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151447) Time Spent: 2h 40m (was: 2.5h) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 2h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151378=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151378 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 21:16 Start Date: 04/Oct/18 21:16 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-427172229 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151378) Time Spent: 2h 10m (was: 2h) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151379=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151379 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 21:16 Start Date: 04/Oct/18 21:16 Worklog Time Spent: 10m Work Description: yifanzou removed a comment on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-427071409 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151379) Time Spent: 2h 20m (was: 2h 10m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151244=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151244 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 15:52 Start Date: 04/Oct/18 15:52 Worklog Time Spent: 10m Work Description: yifanzou removed a comment on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426807980 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151244) Time Spent: 1.5h (was: 1h 20m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151245=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151245 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 15:52 Start Date: 04/Oct/18 15:52 Worklog Time Spent: 10m Work Description: yifanzou removed a comment on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426845708 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151245) Time Spent: 1h 40m (was: 1.5h) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151246=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151246 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 15:52 Start Date: 04/Oct/18 15:52 Worklog Time Spent: 10m Work Description: yifanzou removed a comment on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426845689 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151246) Time Spent: 1h 50m (was: 1h 40m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151247=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151247 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 15:52 Start Date: 04/Oct/18 15:52 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-427071409 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151247) Time Spent: 2h (was: 1h 50m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[
https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151041=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151041
]
ASF GitHub Bot logged work on BEAM-5630:
Author: ASF GitHub Bot
Created on: 04/Oct/18 05:43
Start Date: 04/Oct/18 05:43
Worklog Time Spent: 10m
Work Description: jasonkuster commented on a change in pull request
#6559: [BEAM-5630] add more tests into BigQueryIOReadIT
URL: https://github.com/apache/beam/pull/6559#discussion_r222543106
##
File path:
sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIOReadIT.java
##
@@ -83,13 +94,31 @@ private void runBigQueryIOReadPipeline() {
@Test
public void testBigQueryReadEmpty() throws Exception {
-setupTestEnvironment("empty");
+setupTestEnvironment("empty", false);
runBigQueryIOReadPipeline();
}
@Test
- public void testBigQueryRead1K() throws Exception {
-setupTestEnvironment("1K");
+ public void testBigQueryRead1G() throws Exception {
Review comment:
Missing 1M here as well?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151041)
Time Spent: 1h 20m (was: 1h 10m)
> supplement Bigquery Read IT test cases and blacklist them in post-commit
>
>
> Key: BEAM-5630
> URL: https://issues.apache.org/jira/browse/BEAM-5630
> Project: Beam
> Issue Type: Bug
> Components: testing
>Reporter: yifan zou
>Assignee: yifan zou
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151027=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151027 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 04:34 Start Date: 04/Oct/18 04:34 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426882361 +R @jasonkuster This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151027) Time Spent: 1h 10m (was: 1h) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151009=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151009 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 02:16 Start Date: 04/Oct/18 02:16 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426862928 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151009) Time Spent: 1h (was: 50m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=151002=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151002 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 01:06 Start Date: 04/Oct/18 01:06 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426852321 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 151002) Time Spent: 50m (was: 40m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=150995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-150995 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 00:23 Start Date: 04/Oct/18 00:23 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426845708 Run Python PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 150995) Time Spent: 40m (was: 0.5h) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=150994=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-150994 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 04/Oct/18 00:23 Start Date: 04/Oct/18 00:23 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426845689 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 150994) Time Spent: 0.5h (was: 20m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=150935=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-150935 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 03/Oct/18 21:21 Start Date: 03/Oct/18 21:21 Worklog Time Spent: 10m Work Description: yifanzou commented on issue #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559#issuecomment-426807980 Run Java PostCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 150935) Time Spent: 20m (was: 10m) > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug > Components: testing >Reporter: yifan zou >Assignee: yifan zou >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-5630) supplement Bigquery Read IT test cases and blacklist them in post-commit
[ https://issues.apache.org/jira/browse/BEAM-5630?focusedWorklogId=150934=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-150934 ] ASF GitHub Bot logged work on BEAM-5630: Author: ASF GitHub Bot Created on: 03/Oct/18 21:20 Start Date: 03/Oct/18 21:20 Worklog Time Spent: 10m Work Description: yifanzou opened a new pull request #6559: Do Not Merge. [BEAM-5630] add more tests into BigQueryIOReadIT URL: https://github.com/apache/beam/pull/6559 **Please** add a meaningful description for your change here Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/) | --- | --- | --- | --- | --- | --- Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/) | --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/) [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | --- | --- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 150934) Time Spent: 10m Remaining Estimate: 0h > supplement Bigquery Read IT test cases and blacklist them in post-commit > > > Key: BEAM-5630 > URL: https://issues.apache.org/jira/browse/BEAM-5630 > Project: Beam > Issue Type: Bug >
