Kailash Hassan Dayanand created FLINK-8944:
----------------------------------------------
Summary: Use ListShards for shard discovery in the flink kinesis
connector
Key: FLINK-8944
URL: https://issues.apache.org/jira/browse/FLINK-8944
Project: Flink
Issue Type: Improvement
Reporter: Kailash Hassan Dayanand
Currently the DescribeStream AWS API used to get list of shards is has a
restricted rate limits on AWS. (5 requests per sec per account). This is
problematic when running multiple flink jobs all on same account since each
subtasks calls the Describe Stream. Changing this to ListShards will provide
more flexibility on rate limits as ListShards has a 100 requests per second per
data stream limits.
More details on the mailing list. https://goo.gl/mRXjKh
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
