[
https://issues.apache.org/jira/browse/KAFKA-4606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806263#comment-15806263
]
Ewen Cheslack-Postava commented on KAFKA-4606:
----------------------------------------------
Oh, yeah, I definitely think it'd nice to be able to do. Was just pointing out
that there's a bunch more issues to think about if we were to make it happen.
> make getOffsetsTopicPartitionCount mutable
> --------------------------------------------
>
> Key: KAFKA-4606
> URL: https://issues.apache.org/jira/browse/KAFKA-4606
> Project: Kafka
> Issue Type: Improvement
> Components: consumer
> Reporter: Ryan P
>
> Under certain circumstances users may wish to increase the size of their
> __consumer_offsets topic to spread the load as their consumer load increases.
> The current limitation, in which we try to prevent the topic's expansion
> makes this incredibly difficult. This is done intentionally to reduce the
> risk of unexpected behavior once the group id has been assigned a new
> coordinator.
> Instead Kafka should consider ways to handle this operation more gracefully
> with the following behaviors.
> 1. Calculate the number of __consumer_offsets partitions from the metadata
> cache with each metadata update
> 2. Force a rebalance after the new partitions are added to the topic. *I'm
> sure to take into consideration here as well. Alternatively the operation
> could be documented as risky emphasizing the behavior of the reset policy
> under these conditions.
> Currently the number of partitions for the consumer_offsets topic is fixed.
> This mean you will need to restart each broker to ensure that coordinator
> discovery happens as expected. That seems less than ideal to me for what
> seems to be a fairly artificial limitation of the consumer group
> implementation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
