Re: Leader: -1 on Kafka

2014-04-22 Thread Jun Rao
If you only have 1 replica, when the broker the replica is down, the partition will have no leader. A broker can be down due to soft failures. See https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whypartitionleadersmigratethemselvessometimes ? Thanks, Jun On Mon, Apr 21, 2014 at 1:36 P

Re: Leader: -1 on Kafka

2014-04-21 Thread Joel Koshy
No it is not common/expected. It will be difficult to give you any useful advise without your logs - please send over your state change logs and broker logs. On Mon, Apr 21, 2014 at 11:36:34PM -0500, Kashyap Mhaisekar wrote: > No. Is this error common? How to overcome this? > > Regards, > Kashyap

Re: Leader: -1 on Kafka

2014-04-21 Thread Kashyap Mhaisekar
No. Is this error common? How to overcome this? Regards, Kashyap On Mon, Apr 21, 2014 at 4:02 PM, Joel Koshy wrote: > So only one of the brokers have the data? i.e., with one partition. > > Do you see errors in your broker log around that time? What do you see > in the kafka-state-change.log f

Re: Leader: -1 on Kafka

2014-04-21 Thread Joel Koshy
So only one of the brokers have the data? i.e., with one partition. Do you see errors in your broker log around that time? What do you see in the kafka-state-change.log for that topic/partition? On Mon, Apr 21, 2014 at 03:36:52PM -0500, Kashyap Mhaisekar wrote: > Yes. > I have 3 kafka brokers and

Re: Leader: -1 on Kafka

2014-04-21 Thread Kashyap Mhaisekar
Yes. I have 3 kafka brokers and I have created one topic with 1 partition and 1 replication (default options). All params are default. I was testing it with heavy load and had a storm reading from this and suddenly the messages stop. kafka's list topics script files then show the leader as -1. I g

Re: Leader: -1 on Kafka

2014-04-21 Thread Joel Koshy
Can you describe your set up in more detail and also if you can reproduce this easily? This can happen when none of the replicas for a partition are available, but cannot comment further without details. On Mon, Apr 21, 2014 at 10:54:59AM -0500, Kashyap Mhaisekar wrote: > Hi, > At times, some of k

Leader: -1 on Kafka

2014-04-21 Thread Kashyap Mhaisekar
Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap