Hi Everybody,
I have 2 question regarding the way consumers, consume messages of a
partition.
- * Is it possible to configure Kafka to allow concurrent message
consumption from one partition concurrently? The order is not my concern at
all.*
I couldn't find any way to that by the Group Of Consumer
approach, If it is possible please let me know, If impossible, then let me
know how to address this problem:
For a reason a consumer that is assigned to a partition could
get very slow, and the messages would be processed very slowly. How can I
detect this and stop producing on this slow partition...
- * Suppose I have 5 partitions and 3 consumers and I am using Group of
Consumers model(I had 5 consumers at start but 2 servers crashed), 3
consumers are working busy with their 3 partitions and they never get
finished since the producer produce to their partitions **non-stop and a
little faster than their consumption. What happens to the other 2
partitions that are missing consumers? How the Group of Consumers can
handle this issue?*
*The order is no matter for me, I need a simple configuration that address
my concurrency needs and I need to make sure no message gets into
starvation scenario that never consumed.*
Please let us know, we want to select between Kafka and RabitMQ and we
prefer Kafka because it is growing community and high throughput, But first
we need to address these basic needs.
Thanks,
Reza Aliakabri
