Hi Michal, The rebalance will only be triggered on consumer membership or topic/partition changes. Once triggered it will try to finish the rebalance for at most rebalance.max.retries times, i.e. if it fails it will wait for rebalance.backoff.ms, and then try again until number of retries exhausted. When it happens an exception will be thrown and the consumer may be fallen to a bad state.
Then reason we did not implement automatic restart upon rebalance failures is that it may actually "hide" some issues in the systems that actually caused the rebalance failure. The general design is that if some exception/errors are not expected like the rebalance failures we will let it to possibly hault/kill the instance rather than automatically restart and let it go. Guozhang On Thu, Jul 10, 2014 at 2:24 AM, Michal Michalski < [email protected]> wrote: > Hi, > > Just wondering - is there any reason why rebalance.max.retries is 4 by > default? Is there any good reason why I shouldn't expect my consumers to > keep trying to rebalance for minutes (e.g. 30 retries every 6 seconds), > rather than seconds (4 retries every 2 seconds by default)? > > Also, if my consumer fails to rebalance because of NoNodeException > (org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = > NoNode for /consumers/is-entity-modified-document-group/ids/<something>) > wouldn't that make sense to make Kafka restart it automatically once it > "uses" all the retries attempts? Or recreate the inexistent ZK node like, I > believe, it will happen on consumer restart? > > I'm asking because that kind of errors seem to be "recoverable" ones, but - > if I understand it correctly - with current design they require > implementing additional mechanisms or manual intervention. > > > Kind regards, > Michał > -- -- Guozhang
