Hi Kafka Dev Team/ Users, We have high level consumer group consuming from 32 partitions for a topic. We have been running 48 consumers in this group across multiple servers. We have kept 16 as back-up consumers, and hoping when the consumer dies, meaning when Zookeeper does not have an owner for a particular partition. The back-up consumer will take over. But I do not see this behavior after an active consumer died, the back-up consumer did not pick the partitions. Please let us know what I can do to achieve this. This is very likely scenario when rolling out new code on consumer side (we will be dong incremental code roll out). Please see the exception below. We are using version 0.8 for now.
[mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a], exception during rebalance org.I0Itec.zkclient.exception.ZkNoNodeException: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/mupd_logmon_hb_events/ids/mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a at org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685) at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766) at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761) at kafka.utils.ZkUtils$.readData(Unknown Source) at kafka.consumer.TopicCount$.constructTopicCount(Unknown Source) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(Unknown Source) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(Unknown Source) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown Source) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(Unknown Source) *Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /consumers/mupd_logmon_hb_events/ids/mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a* at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:42) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:921) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:950) at org.I0Itec.zkclient.ZkConnection.readData(ZkConnection.java:103) at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:770) at org.I0Itec.zkclient.ZkClient$9.call(ZkClient.java:766) at org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675) 11 Jun 2014 14:12:16,710 ERROR [mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a_watcher_executor] (kafka.utils.Logging$class.error:?) - [mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a], error during syncedRebalance kafka.common.ConsumerRebalanceFailedException: mupd_logmon_hb_events_sdc-q1-logstream-8-1402448850475-6521f70a can't rebalance *after 4 retries* at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(Unknown Source) at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anon$1.run(Unknown Source) Thanks, Bhavesh