Hi Bryan, So this must be something else since KIP-134 is not in 0.10.2.1, but in the new release 0.11 that hasn't come out yet.
Eno > On 14 Jun 2017, at 21:35, Bryan Baugher <[email protected]> wrote: > > It does seem like we are in a similar situation described in the KIP ( > https://cwiki.apache.org/confluence/display/KAFKA/KIP-134%3A+Delay+initial+consumer+group+rebalance). > For some historical reason we had set our session.timeout.ms to a high > value (5 minutes) which corresponds with the amount of the group would > wait. Lowering helps new consumers join faster but now I'm seeing group > fluctuations so I'll have to continue to dig into why thats happening. Also > worth noting the broker's rebalance timeout seems to switch to > max.poll.interval.ms in 0.10.1. > > On Wed, Jun 14, 2017 at 10:53 AM Bryan Baugher <[email protected]> wrote: > >> While I do have some logs its not trivial to share since the logs are >> across 16 JVMs and a few different hosts. >> >> On Wed, Jun 14, 2017 at 10:34 AM Eno Thereska <[email protected]> >> wrote: >> >>> The delay in that KIP is just 3 seconds, not minutes though, right? Would >>> you have any logs to share? >>> >>> Thanks >>> Eno >>>> On 14 Jun 2017, at 16:14, Bryan Baugher <[email protected]> wrote: >>>> >>>> Our consumer group isn't doing anything stateful and we've seen this >>>> behavior for existing groups as well. It seems like timing could be an >>>> issue, thanks for the information. >>>> >>>> On Tue, Jun 13, 2017 at 7:39 PM James Cheng <[email protected]> >>> wrote: >>>> >>>>> Bryan, >>>>> >>>>> This sounds related to >>>>> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-134%3A+Delay+initial+consumer+group+rebalance >>>>> and https://issues.apache.org/jira/browse/KAFKA-4925. >>>>> >>>>> -James >>>>> >>>>>> On Jun 13, 2017, at 7:02 AM, Bryan Baugher <[email protected]> wrote: >>>>>> >>>>>> The topics already exist prior to starting any of the consumers >>>>>> >>>>>> On Mon, Jun 12, 2017 at 9:35 PM J Pai <[email protected]> >>> wrote: >>>>>> >>>>>>> When are the topics on which these consumer groups consume, created? >>>>>>> >>>>>>> -Jaikiran >>>>>>> On 13-Jun-2017, at 3:18 AM, Bryan Baugher <[email protected]> wrote: >>>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> We are currently experiencing slow startup times for our consumer >>> groups >>>>>>> (16-32 processes for a hundred or more partitions) in the range of >>>>> minutes >>>>>>> (3-15 minutes), where little to no messages are consumed before >>> suddenly >>>>>>> everything just starts working at full speed. >>>>>>> >>>>>>> I'm currently using Kafka 0.9.0.1 but we are in the middle of >>> upgrading >>>>> to >>>>>>> Kafka 0.10.2.1. We also using the newer kafka consumer API and group >>>>>>> management on a simple Apache Storm topology. We don't make use of >>>>> Storm's >>>>>>> kafka spout but instead wrote a simple one ourselves. >>>>>>> >>>>>>> Using the kafka AdminClient I was able to poll for consumer group >>>>> summary >>>>>>> information. What I've found is that the group seems to sit >>>>>>> in PreparingRebalance state for minutes before finally becoming >>> Stable >>>>>>> which then everything starts processing quickly. I've also enabled >>> debug >>>>>>> logging around the consumer's coordinator classes but didn't see >>>>> anything >>>>>>> to indicate the issue. >>>>>>> >>>>>>> I'm hoping that just upgrading to 0.10 or tweaking how we use our >>>>> consumer >>>>>>> in Apache Storm is the problem but are there any pointers on things I >>>>>>> should look at or try? >>>>>>> >>>>>>> Bryan >>>>>>> >>>>>>> >>>>> >>>>> >>> >>>
