I have 12 partitions on 3 different nodes. On Apr 10, 2014 11:41 PM, "Maung Than" <[email protected]> wrote:
> > >>> I have set up kafka 0.8 in 3 servers. I have pushed some data into > these servers. The number of partitions i use is 12, with a replication > factor of 2. > > Are you running multiple broker instances on a single server. > Or your 12 partitions for multiple topics. > I thought you should not have more partitions then the number of brokers > in the cluster for a topic fro better load balancing and failover. > > Thanks, > Maung > > On Apr 9, 2014, at 8:38 PM, Arjun <[email protected]> wrote: > > > Hi , > > > > I have set up kafka 0.8 in 3 servers. I have pushed some data into these > servers. The number of partitions i use is 12, with a replication factor of > 2. > > We use high level consumer to consume messages. we set auto commit to > false, and we do commits almost after each and every successful message > consumption. > > The problem i am facing now is, If i restart my consumers, then the > messages are getting replayed. > > > > I checked the > > > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group group1 > --zkconnect zkhost:zkport --topic testtopic > > > > I see a high lag in some of the partitions and some partitions have none > as the owner. > > To verify consumer rebalance i used the following command > > > > bin/kafka-run-class.sh kafka.tools.VerifyConsumerRebalance --group > group1 --zookeeper.connect zkhost:zkport > > > > but i see errors > > > > [2014-04-10 03:24:37,832] ERROR No owner for partition [testtopic,0] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,861] ERROR No owner for partition [testtopic,1] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,862] ERROR No owner for partition [testtopic,2] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,863] ERROR No owner for partition [testtopic,3] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,864] ERROR No owner for partition [testtopic,4] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,870] ERROR No owner for partition [testtopic,5] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,871] ERROR No owner for partition [testtopic,6] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,872] ERROR No owner for partition [testtopic,7] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,873] ERROR No owner for partition [testtopic,8] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,874] ERROR No owner for partition [testtopic,9] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,928] ERROR No owner for partition [testtopic,9] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,928] ERROR No owner for partition [testtopic,10] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,930] ERROR No owner for partition [testtopic,10] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,930] ERROR No owner for partition [testtopic,11] > (kafka.tools.VerifyConsumerRebalance$) > > [2014-04-10 03:24:37,931] ERROR No owner for partition [testtopic,11] > (kafka.tools.VerifyConsumerRebalance$) > > > > > > I have seen this ( > http://mail-archives.apache.org/mod_mbox/kafka-users/201311.mbox/%3cca+b+uzsnpunichcj6q6yjalccyhnmooiemj_esy0yqucoem...@mail.gmail.com%3E) > in the mail archives and i checked my consumer log for those patterns but > couldn't find any in my logs. > > > > Can some one help me with this. > > > > Thanks > > Arjun Narasimha kota > >
