Re: Leader: -1 on Kafka
No it is not common/expected. It will be difficult to give you any useful advise without your logs - please send over your state change logs and broker logs. On Mon, Apr 21, 2014 at 11:36:34PM -0500, Kashyap Mhaisekar wrote: No. Is this error common? How to overcome this? Regards, Kashyap On Mon, Apr 21, 2014 at 4:02 PM, Joel Koshy jjkosh...@gmail.com wrote: So only one of the brokers have the data? i.e., with one partition. Do you see errors in your broker log around that time? What do you see in the kafka-state-change.log for that topic/partition? On Mon, Apr 21, 2014 at 03:36:52PM -0500, Kashyap Mhaisekar wrote: Yes. I have 3 kafka brokers and I have created one topic with 1 partition and 1 replication (default options). All params are default. I was testing it with heavy load and had a storm reading from this and suddenly the messages stop. kafka's list topics script files then show the leader as -1. I got the same error even with 2 partitions and default replication. Regards, Kashyap On Mon, Apr 21, 2014 at 1:26 PM, Joel Koshy jjkosh...@gmail.com wrote: Can you describe your set up in more detail and also if you can reproduce this easily? This can happen when none of the replicas for a partition are available, but cannot comment further without details. On Mon, Apr 21, 2014 at 10:54:59AM -0500, Kashyap Mhaisekar wrote: Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap
Re: Leader: -1 on Kafka
If you only have 1 replica, when the broker the replica is down, the partition will have no leader. A broker can be down due to soft failures. See https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whypartitionleadersmigratethemselvessometimes ? Thanks, Jun On Mon, Apr 21, 2014 at 1:36 PM, Kashyap Mhaisekar kashya...@gmail.comwrote: Yes. I have 3 kafka brokers and I have created one topic with 1 partition and 1 replication (default options). All params are default. I was testing it with heavy load and had a storm reading from this and suddenly the messages stop. kafka's list topics script files then show the leader as -1. I got the same error even with 2 partitions and default replication. Regards, Kashyap On Mon, Apr 21, 2014 at 1:26 PM, Joel Koshy jjkosh...@gmail.com wrote: Can you describe your set up in more detail and also if you can reproduce this easily? This can happen when none of the replicas for a partition are available, but cannot comment further without details. On Mon, Apr 21, 2014 at 10:54:59AM -0500, Kashyap Mhaisekar wrote: Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap
Leader: -1 on Kafka
Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap
Re: Leader: -1 on Kafka
No. Is this error common? How to overcome this? Regards, Kashyap On Mon, Apr 21, 2014 at 4:02 PM, Joel Koshy jjkosh...@gmail.com wrote: So only one of the brokers have the data? i.e., with one partition. Do you see errors in your broker log around that time? What do you see in the kafka-state-change.log for that topic/partition? On Mon, Apr 21, 2014 at 03:36:52PM -0500, Kashyap Mhaisekar wrote: Yes. I have 3 kafka brokers and I have created one topic with 1 partition and 1 replication (default options). All params are default. I was testing it with heavy load and had a storm reading from this and suddenly the messages stop. kafka's list topics script files then show the leader as -1. I got the same error even with 2 partitions and default replication. Regards, Kashyap On Mon, Apr 21, 2014 at 1:26 PM, Joel Koshy jjkosh...@gmail.com wrote: Can you describe your set up in more detail and also if you can reproduce this easily? This can happen when none of the replicas for a partition are available, but cannot comment further without details. On Mon, Apr 21, 2014 at 10:54:59AM -0500, Kashyap Mhaisekar wrote: Hi, At times, some of kafka topics end up showing the leader as -1. After this, the messages dont get added to the topic nor consumed. I tried digging into why the leader turns -1 *(leader: -1)* Is there a reason why this happens and how it can be resolved? Regards, kashyap
