Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/2081
Hi @rmetzger,
Thanks for letting me know. However, I'd like to close this PR for now for
the following reasons:
1. The new shard-to-subtask assignment logic introduced with this change
will actually need to be moved again to run() as part of implementing Kinesis
reshard handling [FLINK-3231](https://issues.apache.org/jira/browse/FLINK-3231).
2. I've testing this change a bit more on Kinesis streams with high shard
counts, and it seems like the implementation needs more guarantee on that all
subtasks will be able to get the shard list without failing with Amazon's
LimitExceededException even after 3 retries. Since the implementation for
FLINK-3231 will have a separate thread that polls for changes in the shard
list, I'd like to strengthen this guarantee as part of FLINK-3231's PR.
I'm almost done with FLINK-3231, and will reopen a PR to resolve FLINK-3231
and FLINK-4020 together. I'll keep you updated!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
