No sure. How big is your state? On rebalance, state stores might move from one machine to another. To recreate the store on the new machine the underlying changelog topic must be read. This can take some time -- an hour seems quite long though...
To avoid long state recreation periods Kafka Streams support standby task. Try to enable those via StreamsConfig: "num.standby.replicas" http://docs.confluent.io/current/streams/developer-guide.html#optional-configuration-parameters Also check out this section of the docs: http://docs.confluent.io/3.1.1/streams/architecture.html#fault-tolerance -Matthias On 12/11/16 3:14 AM, Gerrit Jansen van Vuuren wrote: > I don't know about speeding up rebalancing, and an hour seems to suggest > something is wrong with zookeeper or you're whole setup maybe. if it > becomes an unsolvable issue for you, you could try > https://github.com/gerritjvv/kafka-fast which uses a different model and > doesn't need balancing or rebalancing. > > disclojure: "Im the library author". > > > > On 11 Dec 2016 11:56 a.m., "Jon Yeargers" <jon.yearg...@cedexis.com> wrote: > > Is there some way to 'help it along'? It's taking an hour or more from when > I start my app to actually seeing anything consumed. > > Plenty of CPU (and IOWait) during this time so I know it's doing > _something_... >
signature.asc
Description: OpenPGP digital signature
