Hi David Sorry to hear about your situation. You need a restart of all the brokers. This is because to determine the group coordinator for a consumer group, a formula is used [1] which is dependent on the number of partitions in the consumer group. Unfortunately, once you change the number of partitions for __consumer_offsets, the new number of partitions is not automatically picked up by this formula since it's only calculated at startup of the broker. Hence, even if you do a rolling restart, you will face periods of uncertainty until all your brokers have finished restarting. You can find more information in the background section of KIP 895 [2].
So, my suggestion would be to perform a rolling restart with an expectation that your cluster may not be able to consume the topic until all brokers have restarted. After the completion of the restart, everything should be ok. Regards, Divij [1] leader_of_partition(abs(hash(CONSUMER_GROUP_NAME)) % __CONSUMER_OFFSETS_PARTITION_COUNT) [2] https://cwiki.apache.org/confluence/display/KAFKA/KIP-895%3A+Dynamically+refresh+partition+count+of+__consumer_offsets Divij Vaidya On Mon, Apr 3, 2023 at 7:42 PM David Ballano Fernandez < dfernan...@demonware.net> wrote: > Hi guys, > > So, we have a Kafka cluster v2.8, and by mistake, we have increased the > partition number from 50 to 52. > > And now we are having some coordinator inconsistencies when consumers try > to consume from the cluster. > > Any advice on how to untangle this mess? would a rolling restart of the > cluster help? or even deleting the consumer_offest topic? > > thanks. >
