Re: Does kafka write key to broker?

2015-07-02 Thread Mohit Kathuria
by default. So for your case, if you do new KeyedMessage[String, String](topic, null /*key*/, partkey, value) then the partkey will be used to determine the partition but not written to broker. Guozhang On Wed, Jun 24, 2015 at 9:45 AM, Mohit Kathuria mkathu...@sprinklr.com wrote

Re: Does kafka write key to broker?

2015-06-24 Thread Mohit Kathuria
in the KeyedMessage will be null. Hope this helps! Thanks, Liquan On Tue, Jun 23, 2015 at 8:18 AM, Mohit Kathuria mkathu...@sprinklr.com wrote: Hi, We are using kafka 0.8.1.1 in our production cluster. I recently started specifying key as the message itself. I just realised that the key is also

Does kafka write key to broker?

2015-06-23 Thread Mohit Kathuria
, just out of anxiety, I want to know whether we can turn off writing the key to the broker. Any configuration I can change to achieve this? -Thanks, Mohit Kathuria

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2015-04-20 Thread Mohit Kathuria
, Dec 22, 2014 at 6:51 PM, Neha Narkhede n...@confluent.io wrote: Can you share a reproducible test case? On Tue, Dec 9, 2014 at 7:11 AM, Mohit Kathuria mkathu...@sprinklr.com wrote: Neha, The same issue reoccured with just 2 consumer processes

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2014-12-16 Thread Mohit Kathuria
Any suggestions what might be going on here. We are very much blinded here and our application is getting effected due to this. -Mohit On Tue, Dec 9, 2014 at 8:41 PM, Mohit Kathuria mkathu...@sprinklr.com wrote: Neha, The same issue reoccured with just 2 consumer processes. The exception

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2014-12-09 Thread Mohit Kathuria
a watch registered. Do you have a way to try this on zk 3.3.4? I would recommend you try the wchp suggestion as well. On Fri, Nov 7, 2014 at 6:07 AM, Mohit Kathuria mkathu...@sprinklr.com wrote: Hi all, Can someone help here. We are getting constant rebalance failure each time

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2014-11-07 Thread Mohit Kathuria
Hi all, Can someone help here. We are getting constant rebalance failure each time a consumer is added beyond a certain number. Did quite a lot of debugging on this and still not able to figure out the pattern. -Thanks, Mohit On Mon, Nov 3, 2014 at 10:53 PM, Mohit Kathuria mkathu

Fwd: Rebalance not happening even after increasing max retries causing conflict in ZK

2014-11-03 Thread Mohit Kathuria
Neha, Looks like an issue with the consumer rebalance not able to complete successfully. We were able to reproduce the issue on topic with 30 partitions, 3 consumer processes(p1,p2 and p3), properties - 40 rebalance.max.retries and 1(10s) rebalance.backoff.ms. Before the process p3 was

Re: Rebalance not happening even after increasing max retries causing conflict in ZK

2014-11-03 Thread Mohit Kathuria
AM, Mohit Kathuria mkathu...@sprinklr.com wrote: Dear Experts, We recently updated to kafka v0.8.1.1 with zookeeper v3.4.5. I have of topic with 30 partitions and 2 replicas. We are using High level consumer api. Each consumer process which is a storm topolofy has 5 streams which

Rebalance not happening even after increasing max retries causing conflict in ZK

2014-10-20 Thread Mohit Kathuria
Dear Experts, We recently updated to kafka v0.8.1.1 with zookeeper v3.4.5. I have of topic with 30 partitions and 2 replicas. We are using High level consumer api. Each consumer process which is a storm topolofy has 5 streams which connects to 1 or more partitions. We are not using storm's