Re: Kafka 0.8 automatically changing leadership

2014-08-06 Thread Marcos Juarez
Thank you Joel, will go thorough those docs and make sure our settings are appropriate on these instances. Marcos Juarez On Tue, Aug 5, 2014 at 5:58 PM, Joel Koshy wrote: > The session expirations (in the log you pasted) lead to the broker > losing its registration from zookeeper (which trigge

Re: Kafka 0.8 automatically changing leadership

2014-08-05 Thread Joel Koshy
The session expirations (in the log you pasted) lead to the broker losing its registration from zookeeper (which triggers the 'broker failure callback' in the controller) - that causes leader election and leaders to move. Session expirations are typically due to GC so you can take a look at https:/

Re: Kafka 0.8 automatically changing leadership

2014-08-05 Thread Marcos Juarez
Joel, Thanks for responding. I see no "Broker failure callback" messages in the logs. However, I did find this: [2014-08-01 16:47:19,866] INFO Client session timed out, have not heard from server in 4213ms for sessionid 0x143f9e2c9956ee0, closing socket connection and attempting reconnect (org

Re: Kafka 0.8 automatically changing leadership

2014-08-01 Thread Joel Koshy
> Leadership moves automatically for at least a few of the topics, which > never happens when we run them on our prod, non-AWS hardware. This causes Under normal operation (i.e., without broker failures) leadership should not move. Leader changes occur when brokers fail - due to GC, controlled s

Kafka 0.8 automatically changing leadership

2014-08-01 Thread Marcos Juarez
Hi, We have a Kafka 0.8 cluster in a test environment (in this case, on AWS EC2 nodes). Even though we've tried to run very little load on this test cluster, it seems like the instances can't even keep up with that. Leadership moves automatically for at least a few of the topics, which never hap