[ https://issues.apache.org/jira/browse/FLINK-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346999#comment-16346999 ]
ASF GitHub Bot commented on FLINK-8516: --------------------------------------- Github user tzulitai commented on a diff in the pull request: https://github.com/apache/flink/pull/5393#discussion_r165084584 --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/FlinkKinesisConsumer.java --- @@ -93,6 +93,12 @@ /** User supplied deserialization schema to convert Kinesis byte messages to Flink objects. */ private final KinesisDeserializationSchema<T> deserializer; + /** + * The function that determines which subtask a shard should be assigned to. + */ + // TODO: instead of the property, use a factory method that would allow subclass to access source context? --- End diff -- Please see my comments below. > FlinkKinesisConsumer does not balance shards over subtasks > ---------------------------------------------------------- > > Key: FLINK-8516 > URL: https://issues.apache.org/jira/browse/FLINK-8516 > Project: Flink > Issue Type: Bug > Components: Kinesis Connector > Affects Versions: 1.4.0, 1.3.2, 1.5.0 > Reporter: Thomas Weise > Assignee: Thomas Weise > Priority: Major > > The hash code of the shard is used to distribute discovered shards over > subtasks round robin. This works as long as shard identifiers are sequential. > After shards are rebalanced in Kinesis, that may no longer be the case and > the distribution become skewed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)