For the purpose of a unit/integration test for Spring XD, I am creating several consumers (in the same group) in quick succession. With just 3 consumers, this triggers 2 rebalances that (on some machines) can't be dealt with in the default 4*2000ms and fails.
I have created a simple use case that reproduces this out of the context of Spring XD. It can be found in the main() method of [1]. If it does not fail on your machine, I believe bumping the number of creations to 4 or 5 may do it.. So, I have a couple of questions: - on my machine, when using various combinations of rebalance.backoff.ms and rebalance.max.retries, this failure always seems to happen after 30s, whatever the combination. On some other (more powerful) machines, it seems to never fail. Is this actually cpu bound? 30s sounds a lot like twice the default tick, so is this at all related to an ephemeral node timing out? - given that this is for the purposes of an integration test, is there any other parameter that I could tweak to have the system settle down faster? - Is this something that is likely to change with the 0.9 rewrite (I saw that the current de-centralized rebalancing mechanics are the cause of other issues)? As a workaround, I tried waiting for nodes to re-appear in ZooKeeper (code at [2]), but this is still very slow (not to mention that this would be very intrusive to the tests I'm trying to write) Lastly, I should mention that I do need to create 3 consumers sequentially (as opposed to say, use a 3 threads consumer). The test in question simply happens to mimic the creation of consumers that may well be on 3 separate machines Best, [1] https://gist.github.com/ericbottard/91aa9ee114c6091e5b7b [2] https://gist.github.com/ericbottard/de87f1edc8eee2b9bee5 -- Eric Bottard
