[
https://issues.apache.org/jira/browse/KAFKA-10283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168100#comment-17168100
]
Guozhang Wang commented on KAFKA-10283:
---------------------------------------
Sure [~high.lee]. I'd suggest you get familiar with the related classes of task
assignment, StreamsPartitionAssignor, HighAvailabilityAssignor, especially the
two-phase logic of 1) first assign across clients, and then 2) within each
client assign across its consumers. And then you can propose a way to
consolidate the data structures usage for these two phases.
> Consolidate client-level and consumer-level assignment within ClientState
> -------------------------------------------------------------------------
>
> Key: KAFKA-10283
> URL: https://issues.apache.org/jira/browse/KAFKA-10283
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Guozhang Wang
> Priority: Major
> Labels: newbie++
>
> In StreamsPartitionAssignor, we do a two-level assignment, one on the
> client-level, and then after the assignment is done we further decide within
> the client how to distributed among consumers if there are more.
> The {{ClientState}} class is used for book-keeping the assigned tasks,
> however it is only used for the first level, while for the second level it is
> done outside of the class and we only keep track of the results in a few maps
> for logging purposes. This leaves us with a bunch of hierarchical maps, e.g.
> some on the client level and some on the consumer level.
> We would like to consolidate some of these maps into a single data structure
> for better keeping track of the assignment information, and also for less bug
> vulnerability causing the assignment information to be inconsistent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
