[
https://issues.apache.org/jira/browse/KAFKA-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504234#comment-16504234
]
Matthias J. Sax commented on KAFKA-6995:
----------------------------------------
In current code base it's set hard-code as Consumer config:
[https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java#L710]
The idea is to reduce number or rebalances. If the consumer sends a leave group
request (ie, before the change) when Streams is closed, an immediate rebalance
is triggered. If the instance is restarted, a second rebalance is triggered.
Thus, a single rolling bounce requires two rebalances (ie, a store is migrated
again and migrated back).
With the change, no leave group request is sent, and thus, when the instance is
stopped nothing happens initially. If the restart happens quickly enough (ie,
before the consumer times out), there will only be a single rebalance for one
rolling bounce. Furthermore, the newly started instance will detect the local
store and thus, the store will not be migrated at all in the single rebalance.
In summary, if you bounce a `N` instance one-by-one, you will get `N`
rebalances without any task/store migration at all.
Does this make sense?
Because the config reduces the number of rebalances, I am still not sure if I
can follow you explanation. Why do you want to disable it, and why this does
help for your case to reduce the number of rebalances if you disable it? You
should get more rebalances, ie, 2xN instead of N.
> Make config "internal.leave.group.on.close" public
> --------------------------------------------------
>
> Key: KAFKA-6995
> URL: https://issues.apache.org/jira/browse/KAFKA-6995
> Project: Kafka
> Issue Type: Improvement
> Components: consumer, streams
> Reporter: Boyang Chen
> Assignee: Boyang Chen
> Priority: Major
> Labels: needs-kip
>
> We are proposing to make the config "internal.leave.group.on.close" public.
> The reason is that for heavy state application the sticky assignment won't
> work because each stream worker will leave group during rolling restart, and
> there is a possibility that some members are left and rejoined while others
> are still awaiting restart. This would then cause multiple rebalance because
> after the ongoing rebalance is done, we are expecting late members to rejoin
> and move state from `stable` to `prepareBalance`. To solve this problem,
> heavy state application needs to use this config to avoid member list update,
> so that at most one rebalance will be triggered at a proper time when all the
> members are rejoined during rolling restart. This should just be one line
> change.
> Code here:
> * <code>internal.leave.group.on.close</code>
> * Whether or not the consumer should leave the group on close. If set to
> <code>false</code> then a rebalance
> * won't occur until <code>session.timeout.ms</code> expires.
> *
> * <p>
> * Note: this is an internal configuration and could be changed in the future
> in a backward incompatible way
> *
> */
> static final String LEAVE_GROUP_ON_CLOSE_CONFIG =
> "internal.leave.group.on.close";
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
