[
https://issues.apache.org/jira/browse/KAFKA-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-19044:
------------------------------------
Component/s: streams
> Handle tasks that are not present in the current topology
> ---------------------------------------------------------
>
> Key: KAFKA-19044
> URL: https://issues.apache.org/jira/browse/KAFKA-19044
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: Lucas Brutschy
> Priority: Major
>
> A heartbeat might be sent to the group coordinator, claiming to own tasks
> that we do not know about. This may happen for a variety of reasons. We need
> some logic to handle those requests. In KIP-1071, we propose to return
> `INVALID_REQUEST` error whenever this happens, effectively letting the
> clients crash. This is safe, and may be a valid approach for the first
> version of the protocol.
> This behavior will, however, make topology updates impossible. [~cadonna]
> proposed to only check that owned tasks match our set of expected tasks if
> the topology epochs between the group and the client match. However, even if
> we let clients crash only when they use the same topology epoch, that would
> mean that scaling down the number of input partitions for a stateless
> application would become impossible without crashing the applications.
> Long term, I am wondering if it isn't a better course of action to just
> ignore tasks that we do not expect. After all, the client is just saying what
> he thinks he owns. If we'd just ignore those tasks, the next target
> assignment would not include the tasks, so the client would automatically be
> instructed to revoke whatever tasks it thinks it currently has.
> The aim of this ticket is to implement a check and a behavior for the first
> version of the protocol, and create follow-up ticket(s) for any extensions
> for partition scaling and topology updating that we may want to add.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
