Hi Guozhang, I have done some testing with various values of heartbeat.interval.ms and they don't seem to have any influence on the error messages. Running kafka-consumer-groups also continues to return that the consumer groups does not exists or is rebalancing. Do you have any suggestions to how I could debug this further?
Regards, Martin On 25 November 2015 at 18:37, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Martin, > > It seems your consumer's heartbeat.interval.ms config value is too small > (default is 3 seconds) for your environment, consider increasing it and see > if this issue goes away. > > At the same time, we have some better error handling fixes in trunk which > will be included in the next point release. > > https://issues.apache.org/jira/browse/KAFKA-2860 > > Guozhang > > > > On Wed, Nov 25, 2015 at 6:54 AM, Martin Skøtt < > martin.sko...@falconsocial.com> wrote: > > > Hi, > > > > I'm experiencing some very strange issues with 0.9. I get these log > > messages from the new consumer: > > > > [main] ERROR > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error > > ILLEGAL_GENERATION occurred while committing offsets for group > > aaa-bbb-reader > > [main] WARN > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator > > - Auto offset commit failed: Commit cannot be completed due to group > > rebalance > > [main] ERROR > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error > > ILLEGAL_GENERATION occurred while committing offsets for group > > aaa-bbb-reader > > [main] WARN > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator > > - Auto offset commit failed: > > [main] INFO > org.apache.kafka.clients.consumer.internals.AbstractCoordinator > > - Attempt to join group aaa-bbb-reader failed due to unknown member id, > > resetting and retrying. > > > > And this in the broker log: > > [2015-11-25 15:41:01,542] INFO [GroupCoordinator 0]: Preparing to > > restabilize group aaa-bbb-reader with old generation 1 > > (kafka.coordinator.GroupCoordinator) > > [2015-11-25 15:41:01,544] INFO [GroupCoordinator 0]: > > Group aaa-bbb-reader generation 1 is dead and removed > > (kafka.coordinator.GroupCoordinator) > > [2015-11-25 15:41:13,474] INFO [GroupCoordinator 0]: Preparing to > > restabilize group aaa-bbb-reader with old generation 0 > > (kafka.coordinator.GroupCoordinator) > > [2015-11-25 15:41:13,475] INFO [GroupCoordinator 0]: Stabilized > > group aaa-bbb-reader generation 1 (kafka.coordinator.GroupCoordinator) > > [2015-11-25 15:41:13,477] INFO [GroupCoordinator 0]: Assignment received > > from leader for group aaa-bbb-reader for generation 1 > > (kafka.coordinator.GroupCoordinator) > > [2015-11-25 15:41:43,478] INFO [GroupCoordinator 0]: Preparing to > > restabilize group aaa-bbb-reader with old generation 1 > > (kafka.coordinator.GroupCoordinator) > > [2015-11-25 15:41:43,478] INFO [GroupCoordinator 0]: > > Group aaa-bbb-reader generation 1 is dead and removed > > (kafka.coordinator.GroupCoordinator) > > > > When this happens the kafka-consumer-groups describe command keeps saying > > that the group no longer exists or is rebalancing. What is probably even > > worse is that my consumers appears to be looping constantly through > > everything written to the topics!? > > > > Does anyone have any input on what might be happening? > > > > I'm running 0.9 locally on my laptop using one Zookeeper and one broker, > > both using the configuration provided in the distribution. I have 13 > topics > > with two partitions each and a replication factor of 1. I run one > producer > > and once consumer also on the same machine. > > > > -- > > Martin Skøtt > > > > > > -- > -- Guozhang >
