I am running Kafka on Kubernetes using the Kafka Strimzi operator. I am using
incremental sticky rebalance strategy by configuring my consumers with the
following:
ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,
org.apache.kafka.clients.consumer.CooperativeStickyAssignor.class.getName()
Each time I scale consumers in my consumer group all existing consumer in the
group generate the following exception
Exception in thread "main"
org.apache.kafka.common.errors.RebalanceInProgressException: Offset commit
cannot be completed since the consumer is undergoing a rebalance for auto
partition assignment. You can try completing the rebalance by calling poll()
and then retry the operation
Any idea on what caused this exception and/or how to resolve it?
Thank you.
