[
https://issues.apache.org/jira/browse/KAFKA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728209#comment-13728209
]
Joel Koshy commented on KAFKA-984:
----------------------------------
Thanks for the patch - this will help a *lot* especially for mirroring.
However, I share Jun's concern about making such a non-trivial change to
0.8. In any event, here are some comments on
scala.kafka.consumer.ZookeeperConsumerConnector
- We should definitely abstract out the common code - syncedPartialRebalance
and WildcardStreamsHandler. I think with some thought we can refactor it
or we end up with copies of relatively complex code.
- The filters on lines 432/433 will not have any effect (I think) since the
maps are immutable. You should probably apply the filter on assignments on
lines 428/429. So metadata for other topics will be fetched unnecessarily,
and fetchers for other topics may be stopped unnecessarily.
- Also, there are topic variables inside the method that shadow the
parameter which makes it harder to determine which variable is in effect
for which scope.
- Logging can be improved/made more concise: few typos and inconsistencies
in capitalization.
- Why do this only if # added topics == 1? Can accept a list of topics to
rebalance for instead right? I do see your note on Sriram's comments, but
I don't see it in this jira. Can you include those comments?
> Avoid a full rebalance in cases when a new topic is discovered but
> container/broker set stay the same
> -----------------------------------------------------------------------------------------------------
>
> Key: KAFKA-984
> URL: https://issues.apache.org/jira/browse/KAFKA-984
> Project: Kafka
> Issue Type: Bug
> Reporter: Guozhang Wang
> Assignee: Guozhang Wang
> Fix For: 0.8
>
> Attachments: KAFKA-984.v1.patch, KAFKA-984.v2.patch,
> KAFKA-984.v2.patch
>
>
> Currently a full rebalance will be triggered on high level consumers even
> when just a new topic is added to ZK. Better avoid this behavior but only
> rebalance on this newly added topic.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
