[ https://issues.apache.org/jira/browse/BEAM-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexey Romanenko resolved BEAM-9476. ------------------------------------ Fix Version/s: 2.21.0 Resolution: Fixed > KinesisIO DescribeStream transient errors are not retried > --------------------------------------------------------- > > Key: BEAM-9476 > URL: https://issues.apache.org/jira/browse/BEAM-9476 > Project: Beam > Issue Type: Bug > Components: io-java-kinesis > Reporter: Aaron Meihm > Assignee: Alexey Romanenko > Priority: Major > Fix For: 2.21.0 > > Time Spent: 5h > Remaining Estimate: 0h > > During Kinesis stream setup {{DescribeStream}} is used in {{listShards}}. > This API call has quota limits that can become problematic when attempting to > configure multiple Kinesis streams in the same AWS account. AWS currently > limits this call by default to 10 times per second for a given account. With > multiple streams, this can be hit and results in a {{RuntimeException}} being > thrown immediately upon the first error. > Ideally the rate limited call can be retried a number of times before giving > up instead of failing immediately. > I have a work in progress PR that resolves this issue at > https://github.com/apache/beam/pull/10973. -- This message was sent by Atlassian Jira (v8.3.4#803005)