>>> I have set up kafka 0.8 in 3 servers. I have pushed some data into these >>> servers. The number of partitions i use is 12, with a replication factor of >>> 2.
Are you running multiple broker instances on a single server. Or your 12 partitions for multiple topics. I thought you should not have more partitions then the number of brokers in the cluster for a topic fro better load balancing and failover. Thanks, Maung On Apr 9, 2014, at 8:38 PM, Arjun <[email protected]> wrote: > Hi , > > I have set up kafka 0.8 in 3 servers. I have pushed some data into these > servers. The number of partitions i use is 12, with a replication factor of 2. > We use high level consumer to consume messages. we set auto commit to false, > and we do commits almost after each and every successful message consumption. > The problem i am facing now is, If i restart my consumers, then the messages > are getting replayed. > > I checked the > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group group1 > --zkconnect zkhost:zkport --topic testtopic > > I see a high lag in some of the partitions and some partitions have none as > the owner. > To verify consumer rebalance i used the following command > > bin/kafka-run-class.sh kafka.tools.VerifyConsumerRebalance --group group1 > --zookeeper.connect zkhost:zkport > > but i see errors > > [2014-04-10 03:24:37,832] ERROR No owner for partition [testtopic,0] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,861] ERROR No owner for partition [testtopic,1] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,862] ERROR No owner for partition [testtopic,2] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,863] ERROR No owner for partition [testtopic,3] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,864] ERROR No owner for partition [testtopic,4] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,870] ERROR No owner for partition [testtopic,5] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,871] ERROR No owner for partition [testtopic,6] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,872] ERROR No owner for partition [testtopic,7] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,873] ERROR No owner for partition [testtopic,8] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,874] ERROR No owner for partition [testtopic,9] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,928] ERROR No owner for partition [testtopic,9] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,928] ERROR No owner for partition [testtopic,10] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,930] ERROR No owner for partition [testtopic,10] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,930] ERROR No owner for partition [testtopic,11] > (kafka.tools.VerifyConsumerRebalance$) > [2014-04-10 03:24:37,931] ERROR No owner for partition [testtopic,11] > (kafka.tools.VerifyConsumerRebalance$) > > > I have seen this ( > http://mail-archives.apache.org/mod_mbox/kafka-users/201311.mbox/%3cca+b+uzsnpunichcj6q6yjalccyhnmooiemj_esy0yqucoem...@mail.gmail.com%3E) > in the mail archives and i checked my consumer log for those patterns but > couldn't find any in my logs. > > Can some one help me with this. > > Thanks > Arjun Narasimha kota
