[
https://issues.apache.org/jira/browse/BEAM-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711129#comment-15711129
]
ASF GitHub Bot commented on BEAM-1033:
--------------------------------------
GitHub user markflyhigh opened a pull request:
https://github.com/apache/incubator-beam/pull/1479
[BEAM-1033] Bigquery Verifier Retry When Query Failed
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [x] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [x] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
- Add retry when query job failed.
- check QueryResponse.jobComplete instead of rows. reference from
https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/query#response
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/markflyhigh/incubator-beam
Add-retry-when-bq-response-is-failed
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/1479.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1479
----
commit 6c5a07f1949146d3fc4aed7fa1cfc55863fcdfb2
Author: Mark Liu <mark...@markliu-macbookpro.roam.corp.google.com>
Date: 2016-12-01T06:20:12Z
[BEAM-1033] Bigquery Verifier Retry When Query Failed
----
> WindowedWordCountIT is flaky
> ----------------------------
>
> Key: BEAM-1033
> URL: https://issues.apache.org/jira/browse/BEAM-1033
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Pei He
> Assignee: Mark Liu
>
> Jenkins link:
> https://builds.apache.org/job/beam_PreCommit_MavenVerify/5145/console
> Running org.apache.beam.examples.WindowedWordCountIT
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 304.282 sec
> <<< FAILURE! - in org.apache.beam.examples.WindowedWordCountIT
> testWindowedWordCountInBatch(org.apache.beam.examples.WindowedWordCountIT)
> Time elapsed: 304.282 sec <<< FAILURE!
> java.lang.AssertionError:
> Expected: Expected checksum is (cd5b52939257e12428a9fa085c32a84dd209b180)
> but: Invalid BigQuery response:
> {"jobComplete":false,"jobReference":{"jobId":"job_0STNX_OD83tQOzo6MvmqXCrk61U","projectId":"apache-beam-testing"},"kind":"bigquery#queryResponse"}
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
> at
> org.apache.beam.runners.dataflow.testing.TestDataflowRunner.run(TestDataflowRunner.java:164)
> at
> org.apache.beam.runners.dataflow.testing.TestDataflowRunner.run(TestDataflowRunner.java:93)
> at
> org.apache.beam.runners.dataflow.testing.TestDataflowRunner.run(TestDataflowRunner.java:61)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:179)
> at
> org.apache.beam.examples.WindowedWordCount.main(WindowedWordCount.java:224)
> at
> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountPipeline(WindowedWordCountIT.java:88)
> at
> org.apache.beam.examples.WindowedWordCountIT.testWindowedWordCountInBatch(WindowedWordCountIT.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:393)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Results :
> Failed tests:
>
> WindowedWordCountIT.testWindowedWordCountInBatch:59->testWindowedWordCountPipeline:88
>
> Expected: Expected checksum is (cd5b52939257e12428a9fa085c32a84dd209b180)
> but: Invalid BigQuery response:
> {"jobComplete":false,"jobReference":{"jobId":"job_0STNX_OD83tQOzo6MvmqXCrk61U","projectId":"apache-beam-testing"},"kind":"bigquery#queryResponse"}
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
