[ https://issues.apache.org/jira/browse/FLINK-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flink Jira Bot updated FLINK-6503: ---------------------------------- Priority: Minor (was: Major) > Refactor KinesisDataFetcher to separate concerns for shard discovery > -------------------------------------------------------------------- > > Key: FLINK-6503 > URL: https://issues.apache.org/jira/browse/FLINK-6503 > Project: Flink > Issue Type: Improvement > Components: Connectors / Kinesis > Reporter: Tzu-Li (Gordon) Tai > Priority: Minor > Labels: auto-deprioritized-major > > Currently, shard discovery is done within the {{KinesisDataFetcher}} 's loop. > I propose to extract that from the fetcher and encapsulate that in a separate > class to separate concerns. > As can be seen in > https://github.com/apache/flink/pull/3001/files#diff-0b702244caed9a73b7b3e1dc8a9cbaebR461, > one downside if we do not do this refactor is apparent when adding > rescalability of the Kinesis consumer. That required shards lookup operations > prioir to running the fetcher, which resulted in the need to expose shard > fetching logic external to the {{KinesisDataFetcher}}. This should be > properly done be separating concerns of shard discovery from the fetcher. -- This message was sent by Atlassian Jira (v8.3.4#803005)