Hey Cliff, Are you using the 0.9.0.0 release? We've fixed a few problems in the 0.9.0 branch, some of which might explain the behavior you're seeing. There was one bug in particular which resulted in the consumer not fetching data for a set of partitions after a rebalance.
-Jason On Thu, Jan 7, 2016 at 1:01 PM, Cliff Rhyne <crh...@signal.co> wrote: > Hi, > > I'm testing out some changes with the 0.9.0.0 new KafkaConsumer API. We > re-use the same consumer group ID across different components of the > application (which consume from different topics). One topic is always > being consumed from, the rest are turned on and off. > > If I only run the always-on consumer, I have a very low occurrence rate of > the log message below: > > org.apache.kafka.clients.consumer.internals.AbstractCoordinator - Attempt > to heart beat failed since the group is rebalancing, try to re-join group. > > > If I run both types of consumers, the log message occurs frequently and the > alway-on consumer eventually doesn't succeed in rejoining (I see the > attempt in the logs to rejoin but nothing happens after that). I only have > logs on the client side to work with; there's nothing showing up in the > kafka logs to show why the group's state isn't stable. > > Thanks, > Cliff > > -- > Cliff Rhyne > Software Engineering Lead > e: crh...@signal.co > signal.co > ________________________ > > Cut Through the Noise > > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. Any unauthorized use of this email is strictly prohibited. > ©2015 Signal. All rights reserved. >
