[
https://issues.apache.org/jira/browse/BEAM-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705305#comment-16705305
]
Heejong Lee commented on BEAM-5514:
-----------------------------------
I think there are two different issues in this single ticket. The first issue
is that quotaExceeded error is not properly handled and the second one is that
the parallelism is not properly controlled so that it generates lots of backoff
messages as well as unnecessary loads to the BigQuery backend.
The first issue can be easily fixed by handling quotaExceeded error in the same
way as rateLimitExceeded error. For the second issue, I would suggest that we
can simply use SingleThreadExecutor instead of UnboundedThreadExecutor. It
won't hurt the overall performance much since BigQuery.write already set up 50
shards before writing which means there will be up to 50 concurrent
StreamingWriteFns if possible.
> BigQueryIO doesn't handle quotaExceeded errors properly
> -------------------------------------------------------
>
> Key: BEAM-5514
> URL: https://issues.apache.org/jira/browse/BEAM-5514
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Reporter: Kevin Peterson
> Assignee: Heejong Lee
> Priority: Major
>
> When exceeding a streaming quota for BigQuery insertAll requests, BigQuery
> returns a 403 with reason "quotaExceeded".
> The current implementation of BigQueryIO does not consider this to be a rate
> limited exception, and therefore does not perform exponential backoff
> properly, leading to repeated calls to BQ.
> The actual error is in the
> [ApiErrorExtractor|https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L739]
> class, which is called from
> [BigQueryServicesImpl|https://github.com/GoogleCloudPlatform/bigdata-interop/blob/master/util/src/main/java/com/google/cloud/hadoop/util/ApiErrorExtractor.java#L263]
> to determine how to retry the failure.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
