Hello everyone, I have a very low volume topic that has 2 consumers in the same group. How do I get each consumer to only consume 1 message at a time and if the the first consumer is busy get the other consumer to consume the message?
Currently what I am doing is: First consumer connects to Kafka waits for 300 milliseconds then disconnects, waits for 10 seconds, then reconnects to see if there is a waiting message. The messages kick off a long task on each server, each server can handle multiple tasks up to a limit so first I am trying to balance the tasks across multiple servers and if they are maxed out don't consume any messages. This will give the other server or servers a chances to pickup a message and do the task. I would not disconnect if I can ensure I don't have messages waiting in the queue for a server to consume them without the other servers being able to see them. Thanks for the help... Cheers, Eric Sites
