[
https://issues.apache.org/jira/browse/KAFKA-13132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Gustafson updated KAFKA-13132:
------------------------------------
Affects Version/s: (was: 3.0.0)
> Upgrading to topic IDs in LISR requests has gaps introduced in 3.0
> ------------------------------------------------------------------
>
> Key: KAFKA-13132
> URL: https://issues.apache.org/jira/browse/KAFKA-13132
> Project: Kafka
> Issue Type: Bug
> Reporter: Justine Olshan
> Assignee: Justine Olshan
> Priority: Blocker
> Fix For: 3.0.0
>
>
> With the change in 3.0 to how topic IDs are assigned to logs, a bug was
> inadvertently introduced. Now, topic IDs will only be assigned on the load of
> the log to a partition in LISR requests. This means we will only assign topic
> IDs for newly created topics/partitions, on broker startup, or potentially
> when a partition is reassigned.
>
> In the case of upgrading from an IBP before 2.8, we may have a scenario where
> we upgrade the controller to IBP 3.0 (or even 2.8) last. (Ie, the controller
> is IBP < 2.8 and all other brokers are on the newest IBP) Upon the last
> broker upgrading, we will elect a new controller but its LISR request will
> not result in topic IDs being assigned to logs of existing topics. They will
> only be assigned in the cases mentioned above.
> *Keep in mind, in this scenario, topic IDs will be still be assigned in the
> controller/ZK to all new and pre-existing topics and will show up in
> metadata.* This means we are not ensured the same guarantees we had in 2.8.
> *It is just the LISR/partition.metadata part of the code that is affected.*
>
> The problem is two-fold
> 1. We ignore LISR requests when the partition leader epoch has not increased
> (previously we assigned the ID before this check)
> 2. We only assign the topic ID when we are associating the log with the
> partition in replicamanager for the first time. Though in the scenario
> described above, we have logs associated with partitions that need to be
> upgraded.
>
> We should check the if the LISR request is resulting in a topic ID addition
> and add logic to logs already associated to partitions in replica manager.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
