[
https://issues.apache.org/jira/browse/BEAM-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16860421#comment-16860421
]
Chamikara Jayalath commented on BEAM-7424:
------------------------------------------
There are few things that are planned to do.
(1) Add retry logic for Java and Python SDKs.
(2) For Dataflow runner, plumb throttled time to Dataflow backend to consider
when making autoscaling decisions.
This bug is for (1), It's great if we can get (2) into 2.14 as well but I'm not
sure if timelines will match.
> Retry HTTP 429 errors from GCS w/ exponential backoff when reading data
> -----------------------------------------------------------------------
>
> Key: BEAM-7424
> URL: https://issues.apache.org/jira/browse/BEAM-7424
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp, io-python-gcp, sdk-py-core
> Reporter: Chamikara Jayalath
> Assignee: Heejong Lee
> Priority: Blocker
> Fix For: 2.14.0
>
>
> This has to be done for both Java and Python SDKs.
> Seems like Java SDK already retries 429 errors w/o backoff (please verify):
> [https://github.com/apache/beam/blob/master/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/RetryHttpRequestInitializer.java#L185]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
