[
https://issues.apache.org/jira/browse/KAFKA-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504209#comment-16504209
]
Boyang Chen edited comment on KAFKA-6995 at 6/7/18 4:00 AM:
------------------------------------------------------------
Basically leave group request will make the rebalance faster because we could
detect consumer offline faster. In our case of rolling restart, it is favorable
not to do so because leaving group explicitly will break the existing member
list, which means there are always hosts restarted early and leave the group,
while hosts restarted late will not be considered for rebalance. The incomplete
rebalance will trigger multiple times until the consumer list is stabilized.
Where did Kafka Streams set this config to false? I didn't find that code,
could you elaborate on that? [~mjsax]
was (Author: bchen225242):
Basically leave group request will make the rebalance faster because we could
detect consumer offline faster. In our case of rolling restart, it is favorable
not to do so because leaving group explicitly will break the existing member
list, which means there are always hosts restarted early and leave the group,
while hosts restarted late will not be considered for rebalance. The incomplete
rebalance will trigger multiple times until the consumption is stabilized.
Where did Kafka Streams set this config to false? I didn't find that code,
could you elaborate on that? [~mjsax]
> Make config "internal.leave.group.on.close" public
> --------------------------------------------------
>
> Key: KAFKA-6995
> URL: https://issues.apache.org/jira/browse/KAFKA-6995
> Project: Kafka
> Issue Type: Improvement
> Components: consumer, streams
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
> Labels: needs-kip
>
> We are proposing to make the config "internal.leave.group.on.close" public.
> The reason is that for heavy state application the sticky assignment won't
> work because each stream worker will leave group during rolling restart, and
> there is a possibility that some members are left and rejoined while others
> are still awaiting restart. This would then cause multiple rebalance because
> after the ongoing rebalance is done, we are expecting late members to rejoin
> and move state from `stable` to `prepareBalance`. To solve this problem,
> heavy state application needs to use this config to avoid member list update,
> so that at most one rebalance will be triggered at a proper time when all the
> members are rejoined during rolling restart. This should just be one line
> change.
> Code here:
> * <code>internal.leave.group.on.close</code>
> * Whether or not the consumer should leave the group on close. If set to
> <code>false</code> then a rebalance
> * won't occur until <code>session.timeout.ms</code> expires.
> *
> * <p>
> * Note: this is an internal configuration and could be changed in the future
> in a backward incompatible way
> *
> */
> static final String LEAVE_GROUP_ON_CLOSE_CONFIG =
> "internal.leave.group.on.close";
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
