In addition to the session timeout try increasing the request timeout as well. We had similar issue and resolved it by increasing the timeouts. As per my understanding, If you have complex topology then it will take some time for kafka brokers to create the tasks and assign them to consumers. In the mean time if any consumer try to join the group it will timeout due to lower timeout values. Increasing the timeouts will give enough time for the brokers to assign the tasks to consumers properly.
request.timeout.ms=60000 Thanks Thameem > On Jul 11, 2019, at 7:56 PM, Aravind Dongara <adong...@yahoo.com.INVALID> > wrote: > > Hi Boyang, > > Thanks for the quick response. > We are on version 2.2.0. > > We are using the following properties on KStreams/consumer: > session.timeout.ms=15000 > heartbeat.interval.ms=3000 > > I was wondering if a member might leak if it satisfies “shouldKeepAlive” > condition in "onExpireHeartbeat" and the consumer restarts or goes down right > after that, before next heartbeat is sent. > > Thanks > Aravind > > >> On Jul 10, 2019, at 10:40 PM, Boyang Chen <reluctanthero...@gmail.com> wrote: >> >> Hey Aravind, >> >> If your client/broker just upgraded to 2.3, Jason has filed a blocker for >> 2.3: https://issues.apache.org/jira/browse/KAFKA-8653 >> >> and a fix is on its way: https://github.com/apache/kafka/pull/7072/files >> >> Let me know if you are actually on a different version. >> >> Boyang >> >> On Wed, Jul 10, 2019 at 7:43 PM Aravind Dongara <adong...@yahoo.com.invalid> >> wrote: >> >>> Our kafka streams application is stuck and continuously emits >>> "(Re-)joining group” log message every 5 minutes without making any >>> progress. >>> >>> Kafka-consumer-groups cmd line tool with “—members” option shows lots of >>> stale members, in addition to expected member-ids shown on log msgs on >>> kafka-streams app and broker logs that were failing to join). >>> For some reason these old members didn’t get evicted from members list. >>> Looks like this is preventing the GroupCordinator from reaching >>> “CompletingRebalance” state. >>> >>> Restart of Kafka streams app didn’t help either, it just replaced the >>> newer member-ids; but the old stale member-ids are still present in the >>> members-list. >>> >>> Is there any way to resolve this without restarting the broker hosting the >>> GroupCoordinator for this group. >>> >>> Thanks >>> Aravind >
