Hi Divij, Thanks for your reply! Will get a rolling restart going! On Tue, Apr 4, 2023 at 8:54 AM Divij Vaidya <divijvaidy...@gmail.com> wrote:
> Hi David Sorry to hear about your situation. You need a restart of all the > brokers. This is because to determine the group coordinator for a consumer > group, a formula is used [1] which is dependent on the number of partitions > in the consumer > ZjQcmQRYFpfptBannerStart > This Message Is From an Untrusted Sender > You have not previously corresponded with this sender. > > ZjQcmQRYFpfptBannerEnd > > Hi David > > Sorry to hear about your situation. You need a restart of all the brokers. > This is because to determine the group coordinator for a consumer group, a > formula is used [1] which is dependent on the number of partitions in the > consumer group. Unfortunately, once you change the number of partitions for > __consumer_offsets, the new number of partitions is not > automatically picked up by this formula since it's only calculated at > startup of the broker. > Hence, even if you do a rolling restart, you will face periods of > uncertainty until all your brokers have finished restarting. You can find > more information in the background section of KIP 895 [2]. > > So, my suggestion would be to perform a rolling restart with an expectation > that your cluster may not be able to consume the topic until all brokers > have restarted. After the completion of the restart, everything should be > ok. > > Regards, > Divij > > [1] leader_of_partition(abs(hash(CONSUMER_GROUP_NAME)) % > __CONSUMER_OFFSETS_PARTITION_COUNT) > [2]https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_KAFKA_KIP-2D895-253A-2BDynamically-2Brefresh-2Bpartition-2Bcount-2Bof-2B-5F-5Fconsumer-5Foffsets&d=DwIFaQ&c=qE8EibqjfXM-zBfebVhd4gtjNZbrDcrKYXvb1gt38s4&r=p-f3AJg4e4Uk20g_16kSyBtabT4JOB-1GIb23_CxD58&m=9DpGX5TXJQUt8QvQvJNrVcwZK1-pYTdpgPjYwQkXkYA752EKHdc2kbLnYC6c9xyj&s=5ncO1fPHJBb_TeHewskIs2vXiZMOexEVhduDZ2cUceY&e= > > > > > Divij Vaidya > > > > On Mon, Apr 3, 2023 at 7:42 PM David Ballano Fernandez > <dfernan...@demonware.net> wrote: > > > Hi guys, > > > > So, we have a Kafka cluster v2.8, and by mistake, we have increased the > > partition number from 50 to 52. > > > > And now we are having some coordinator inconsistencies when consumers try > > to consume from the cluster. > > > > Any advice on how to untangle this mess? would a rolling restart of the > > cluster help? or even deleting the consumer_offest topic? > > > > thanks. > > > >
