Antony Stubbs created KAFKA-6145: ------------------------------------ Summary: Warm up new KS instances before migrating tasks - potentially a two phase rebalance Key: KAFKA-6145 URL: https://issues.apache.org/jira/browse/KAFKA-6145 Project: Kafka Issue Type: New Feature Components: streams Reporter: Antony Stubbs
Currently when expanding the KS cluster, the new node's partitions will be unavailable during the rebalance, which for large states can take a very long time, or for small state stores even more than a few ms can be a deal breaker for micro service use cases. One workaround would be two execute the rebalance in two phases: 1) start running state store building on the new node 2) once the state store is fully populated on the new node, only then rebalance the tasks - there will still be a rebalance pause, but would be greatly reduced -- This message was sent by Atlassian JIRA (v6.4.14#64029)