[
https://issues.apache.org/jira/browse/KAFKA-21072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-21072:
-----------------------------------
Description:
Client metadata logic ignores metadata received with an older epoch. But this
may lead to clients ignoring new partitions in cases of topic re-creation:
* topic is in metadata with N partitions (topicId-1) -> known epochs > 0
* deleted + recreated (topicId-2) with 2 partitions -> epochs back to 0 for
the 2 partitions only
* topic expanded to N partitions (same topicId-2) -> metadata dropped for all
new partitions because they will include epoch 0 older than the last known
leader epoch (which is the stale one from before the re-creation)
[https://github.com/apache/kafka/blob/051a1860d55ee96eec2eb2c75ee1d6dd7c5183c9/clients/src/main/java/org/apache/kafka/clients/Metadata.java#L593-L595]
This issue seems the same that was fixed with
https://issues.apache.org/jira/browse/KAFKA-12257 for the re-creation case (did
not include the case of the expansion on top of the re-creation)
was:
Client metadata logic ignores metadata received with an older epoch. But this
may lead to clients ignoring new partitions in cases of topic re-creation:
* topic is in metadata with N partitions (topicId-1) -> known epochs > 0
* deleted + recreated (topicId-2) with 2 partitions -> epochs back to 0 for
the 2 partitions only
* topic expanded to N partitions (same topicId-2) -> metadata dropped for all
new partitions because they will include epoch 0 older than the last known
leader epoch (which is the stale one from before the re-creation)
[https://github.com/apache/kafka/blob/051a1860d55ee96eec2eb2c75ee1d6dd7c5183c9/clients/src/main/java/org/apache/kafka/clients/Metadata.java#L593-L595]
This issue is probably an extension to
https://issues.apache.org/jira/browse/KAFKA-12257 that addressed this same
issue but for the full re-creation case (not the expansion on top of the
re-creation)
> Client metadata may ignore expanded partitions after topic re-creation
> ----------------------------------------------------------------------
>
> Key: KAFKA-21072
> URL: https://issues.apache.org/jira/browse/KAFKA-21072
> Project: Kafka
> Issue Type: Bug
> Components: clients, consumer
> Reporter: Lianet Magrans
> Priority: Major
>
> Client metadata logic ignores metadata received with an older epoch. But this
> may lead to clients ignoring new partitions in cases of topic re-creation:
> * topic is in metadata with N partitions (topicId-1) -> known epochs > 0
> * deleted + recreated (topicId-2) with 2 partitions -> epochs back to 0 for
> the 2 partitions only
> * topic expanded to N partitions (same topicId-2) -> metadata dropped for
> all new partitions because they will include epoch 0 older than the last
> known leader epoch (which is the stale one from before the re-creation)
> [https://github.com/apache/kafka/blob/051a1860d55ee96eec2eb2c75ee1d6dd7c5183c9/clients/src/main/java/org/apache/kafka/clients/Metadata.java#L593-L595]
> This issue seems the same that was fixed with
> https://issues.apache.org/jira/browse/KAFKA-12257 for the re-creation case
> (did not include the case of the expansion on top of the re-creation)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
