Udi Meiri created BEAM-7251: ------------------------------- Summary: Testing BigQuery client fails queries if job results aren't immediately available Key: BEAM-7251 URL: https://issues.apache.org/jira/browse/BEAM-7251 Project: Beam Issue Type: Bug Components: io-java-gcp Reporter: Udi Meiri
Adding functionality to fetch query results should solve this issue, which is probably causing test flakiness. Log: May 05, 2019 5:29:19 PM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2019-05-05_17_25_26-4118012232925193147. Found 0 success, 0 failures out of 0 expected assertions. May 05, 2019 5:29:19 PM org.apache.beam.sdk.io.gcp.testing.BigqueryMatcher matchesSafely INFO: Verifying Bigquery data May 05, 2019 5:29:29 PM com.google.cloud.dataflow.testing.DataflowJUnitTestRunner main SEVERE: testE2eBigQueryTornadoesWithStorageApi(org.apache.beam.examples.cookbook.BigQueryTornadoesIT) java.lang.AssertionError: Expected: Expected checksum is (1ab4c7ec460b94bbb3c3885b178bf0e6bed56e1f) but: The query job hasn't completed. Got response: {"jobComplete":false,"jobReference":{"jobId":"job_cZkICLalRsrnivu78BX1y3UwMhIz","location":"US","projectId":"xxx"},"kind":"bigquery#queryResponse"} at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20) at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8) at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:138) at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90) at org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299) at org.apache.beam.examples.cookbook.BigQueryTornadoes.runBigQueryTornadoes(BigQueryTornadoes.java:199) at org.apache.beam.examples.cookbook.BigQueryTornadoesIT.runE2EBigQueryTornadoesTest(BigQueryTornadoesIT.java:70) at org.apache.beam.examples.cookbook.BigQueryTornadoesIT.testE2eBigQueryTornadoesWithStorageApi(BigQueryTornadoesIT.java:95) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at com.google.cloud.dataflow.testing.DataflowJUnitTestRunner.main(DataflowJUnitTestRunner.java:145) Exception in thread "main" java.lang.IllegalStateException: Tests failed, check output logs for details. at com.google.cloud.dataflow.testing.DataflowJUnitTestRunner.main(DataflowJUnitTestRunner.java:154) But checking BQ logs on the console reveals that the query job did run: 2019-05-05 17:29:29.601 PDT Bigquery query queries 2019-05-05 17:29:31.956 PDT Bigquery jobcompleted job_cZkICLalRsrnivu78BX1y3UwMhIz -- This message was sent by Atlassian JIRA (v7.6.3#76005)