Bruno Cadonna created KAFKA-14288:
-------------------------------------
Summary: Processor topology in tasks is updated with internal
intermediate topics without application ID prefix
Key: KAFKA-14288
URL: https://issues.apache.org/jira/browse/KAFKA-14288
Project: Kafka
Issue Type: Bug
Components: streams
Affects Versions: 3.3.1
Reporter: Bruno Cadonna
Updating the input partitions of tasks during assignment handling also updates
the mapping from source nodes to input topics in the processor topology within
the task. The mapping is updated with the topics from the topology metadata.
The topology metadata does not prefix internal intermediate topics with the
application ID. Thus, if a standby task has input partitions from an internal
intermediate topic the update of the mapping in the processor topology leads to
an invalid topology exception during recycling of a standby task to an active
task when the input queues are created. This is because the input topics in the
processor topology and the input partitions of the task do not match because
the former miss the application ID prefix.
The case were standby tasks have input partitions only from internal
intermediate topics can be fixed by checking if the input partitions really
changed before updating the input partitions (see PR
https://github.com/apache/kafka/pull/12730). Unfortunately, a subtopology might
have input partitions subscribed to with a regex additionally to internal
intermediate topics which might still lead to an invalid topology exception
during recycling irrespectively of the aforementioned verification.
This bug might also affect active tasks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
