InvalidMessageSizeException

2013-02-11 Thread Manish Khettry
One of our consumers keeps getting an invalid message size exception. I'm pretty sure that we don't have a message size this big (1.7G). We have two other consumer groups consuming messages from the same Kafka instance happily over the last few days. Since we keep the logs around for a fixed inter

Re: Consumers constantly rebalancing

2013-02-05 Thread Manish Khettry
GC pauses on your client side or due > to > > IO pauses on the zookeeper server side. > > That is the reason increasing the session timeout seems to have helped. > > If this error happens frequently, it will cause your consumer instances > to > > keep rebalancing.

Consumers constantly rebalancing

2013-02-05 Thread Manish Khettry
We are trying to trouble shoot a problem wherein our system just cannot seem to read messages fast enough from Kafka. We are on kafka 0.6 and are using the simple consumer. >From looking at the logs, and we see a lot (almost constant chatty messages) about rebalancing. So for instance every minute