Re: Kafka streams uneven task allocation

2022-02-23 Thread Guozhang Wang
Hello Navneeth, Just to verify some behaviors, could you try 1) not using instance.id config, hence no static members, 2) upgrade to the latest version of Kafka, respectively (i.e. do not do them at the same time) and see if either one of them help with the imbalance issue? On Sun, Feb 20, 2022 a

Re: Kafka streams uneven task allocation

2022-02-20 Thread Luke Chen
Hi Navneeth, To know the reason why there's more than one partition in the same stream task, we should know why the rebalance triggered. That might have to look into the logs. > I have configured standby to be 1 which means there will be one more copy of the state store and warm up by default is

Re: Kafka streams uneven task allocation

2022-02-18 Thread Navneeth Krishnan
Hi Guozhang, Thanks and sorry for the late reply. I'm overriding the GROUP_INSTANCE_ID_CONFIG & APPLICATION_SERVER_CONFIG. Rest all are defaults. Even then I see more than one partition being allocated to the same stream task. Also I have an additional question regarding the replicas. The default

Re: Kafka streams uneven task allocation

2022-02-03 Thread Guozhang Wang
Hello Navneeth, Could you describe how you ended up with more than one partition assigned to the same thread after certain rebalance(s)? Do you override any default config values such as instance.id (for static consumer members), etc? Also I'd suggest upgrading to a newer version --- we just rele

Kafka streams uneven task allocation

2022-02-02 Thread Navneeth Krishnan
Hi All, We are facing an issue with our kafka streams application due to uneven task allocation. There are 100 partitions in the input topic with 100 stream threads processing the data. Everything works well when each task gets assigned with 1 partition. But when more than one partition is assigne