Hello,

We are using a multi-topic approach for partitioning data. Each topic will
follow a pattern naming convention. We want to design our consumer
configuration so that a slow topic will not block a fast topic, as each
topic will consume at a different rate.

However I am seeing an issue where all topics matching the pattern are
assigned to a single consumer. When there are several consumers in the
group with a single topic with many partitions, the partitions are
automatically distributed among the available consumers in the group. I
expected that calling KafkaConsumer#subscribe(Pattern,
ConsumerRebalanceListener) would follow a similar behavior by distributing
the assigned topics among all consumers in the group.

Is this not the case? What is the expected behavior and how would you
recommend implementing this design?

Thank you

-- 
Robert Quinlivan
Software Engineer, Signal

Reply via email to