Not sure if there is an easy way to do what you want. One thing I can think of is to reduce queuedchunks.max and fetch.size to reduce the # of queued messages.
Thanks, Jun On Tue, Aug 13, 2013 at 6:47 PM, Eric Sites <eric.si...@threattrack.com>wrote: > Hello everyone, > > I have a very low volume topic that has 2 consumers in the same group. How > do I get each consumer to only consume 1 message at a time and if the the > first consumer is busy get the other consumer to consume the message? > > Currently what I am doing is: > > First consumer connects to Kafka waits for 300 milliseconds then > disconnects, waits for 10 seconds, then reconnects to see if there is a > waiting message. > > The messages kick off a long task on each server, each server can handle > multiple tasks up to a limit so first I am trying to balance the tasks > across multiple servers and if they are maxed out don't consume any > messages. > > This will give the other server or servers a chances to pickup a message > and do the task. > > I would not disconnect if I can ensure I don't have messages waiting in > the queue for a server to consume them without the other servers being able > to see them. > > Thanks for the help... > > Cheers, > Eric Sites > > >
