[
https://issues.apache.org/jira/browse/KAFKA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18050374#comment-18050374
]
sanghyeok An commented on KAFKA-19424:
--------------------------------------
Hi [~squah-confluent] !
During my investigation of both __consumer_offsets and {{__transaction_state}}
alongside this work,
I realized that this issue extends to all internal topics.
Also, the documentation explicitly discourages modifying the partition count
for internal topics post-deployment.
To improve this behavior, I have prepared a KIP.
(https://cwiki.apache.org/confluence/display/KAFKA/KIP-1268%3A+Prevent+Dynamic+Partition+Expansion+of+Internal+Topics)
I would greatly appreciate your insights if you have the time to review it.
Thanks for your time!
> FindCoordinator inconsistent across brokers when __consumer_offsets partition
> count is increased
> ------------------------------------------------------------------------------------------------
>
> Key: KAFKA-19424
> URL: https://issues.apache.org/jira/browse/KAFKA-19424
> Project: Kafka
> Issue Type: Bug
> Components: group-coordinator
> Reporter: Sean Quah
> Assignee: sanghyeok An
> Priority: Major
>
> {{GroupCoordinatorService}} captures the number of partitions of
> {{\_\_consumer_offsets}} at startup. This is used to map group ids to
> {{\_\_consumer_offsets}} partitions in
> {{GroupCoordinatorService.partitionFor}}.
> When adding partitions to {{\_\_consumer_offsets}},
> {{GroupCoordinatorService}} doesn't update its cached partition count. This
> means that newly started brokers will map group ids to partitions differently
> to existing brokers and FindCoordinator requests can return different results
> depending on which broker the client asks.
> It's proposed to make brokers consistent by updating the cached partition
> count in {{GroupMetadataService.onNewMetadataImage}}.
> We should additionally log a warning when the partition count changes, since
> adding partitions to {{\_\_consumer_offsets}} is not truly supported, as any
> existing group info will remain on the old partition and become un-findable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
