Justine Olshan created KAFKA-13132:
--------------------------------------
Summary: Upgrading to topic IDs in LISR requests has gaps
introduced in 3.0
Key: KAFKA-13132
URL: https://issues.apache.org/jira/browse/KAFKA-13132
Project: Kafka
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Justine Olshan
Assignee: Justine Olshan
With the change in 3.0 to how topic IDs are assigned to logs, a bug was
inadvertently introduced. Now, topic IDs will only be assigned on the load of
the log to a partition in LISR requests. This means we will only assign topic
IDs for newly created topics/partitions, on broker startup, or potentially when
a partition is reassigned.
In the case of upgrading from an IBP before 2.8, we may have a scenario where
we upgrade the controller to IBP 3.0 last. (Ie, the controller is IBP < 2.8 and
all other brokers are on IBP 3.0) Upon the last broker upgrading, we will elect
a new controller but its LISR request may not result in topic IDs being
assigned to logs.
Keep in mind, in this scenario, topic IDs will be assigned to all pre-existing
topics and will show up in metadata. It is just the LISR/partition.metadata
part of the code that is affected. This means we are not ensured the same
guarantees we had in 2.8.
The problem is two-fold
1. We ignore LISR requests when the partition leader epoch has not increased
(previously we assigned the ID before this check)
2. We only assign the topic ID when we are associating the log with the
partition in replicamanager for the first time. Though in the scenario
described above, we have logs associated with partitions that need to be
upgraded.
We should check the if the LISR request is resulting in a topic ID addition and
add logic to logs already associated to partitions in replica manager.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
