[
https://issues.apache.org/jira/browse/KAFKA-6144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navinder Brar updated KAFKA-6144:
---------------------------------
Attachment: image-2019-10-09-20-33-37-423.png
> Allow state stores to serve stale reads during rebalance
> --------------------------------------------------------
>
> Key: KAFKA-6144
> URL: https://issues.apache.org/jira/browse/KAFKA-6144
> Project: Kafka
> Issue Type: New Feature
> Components: streams
> Reporter: Antony Stubbs
> Priority: Major
> Labels: needs-kip
> Attachments: image-2019-10-09-20-33-37-423.png
>
>
> Currently when expanding the KS cluster, the new node's partitions will be
> unavailable during the rebalance, which for large states can take a very long
> time, or for small state stores even more than a few ms can be a deal breaker
> for micro service use cases.
> One workaround is to allow stale data to be read from the state stores when
> use case allows.
> Relates to KAFKA-6145 - Warm up new KS instances before migrating tasks -
> potentially a two phase rebalance
> This is the description from KAFKA-6031 (keeping this JIRA as the title is
> more descriptive):
> {quote}
> Currently reads for a key are served by single replica, which has 2 drawbacks:
> - if replica is down there is a down time in serving reads for keys it was
> responsible for until a standby replica takes over
> - in case of semantic partitioning some replicas might become hot and there
> is no easy way to scale the read load
> If standby replicas would have endpoints that are exposed in StreamsMetadata
> it would enable serving reads from several replicas, which would mitigate the
> above drawbacks.
> Due to the lag between replicas reading from multiple replicas simultaneously
> would have weaker (eventual) consistency comparing to reads from single
> replica. This however should be acceptable tradeoff in many cases.
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
