Hello, We are using a multi-topic approach for partitioning data. Each topic will follow a pattern naming convention. We want to design our consumer configuration so that a slow topic will not block a fast topic, as each topic will consume at a different rate.
However I am seeing an issue where all topics matching the pattern are assigned to a single consumer. When there are several consumers in the group with a single topic with many partitions, the partitions are automatically distributed among the available consumers in the group. I expected that calling KafkaConsumer#subscribe(Pattern, ConsumerRebalanceListener) would follow a similar behavior by distributing the assigned topics among all consumers in the group. Is this not the case? What is the expected behavior and how would you recommend implementing this design? Thank you -- Robert Quinlivan Software Engineer, Signal
