Hi All, I was wondering how an application can be auto-scalable if only a single instance can read from the single Kafka partition and two instances cannot read from the single partition at the same time with the same consumer group.
Suppose there is an application that has 10 instances running on Kubernetes in production at this moment (using the same consumer group) and we have got a Kafka topic with 10 partitions. Due to the increase in load, Kubernetes provision more instance to take the extra load. However, since the maximum number of consumers with the same consumer group can be 10 in this example, no matter how many new instances are created they are not able to address the extra load until partition number increases. Is there any out of the box solution to address this situation? Thanks, Ali
