[jira] [Commented] (KAFKA-12426) Missing logic to create partition.metadata files in RaftReplicaManager

2021-03-12 Thread Jason Gustafson (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300726#comment-17300726
 ] 

Jason Gustafson commented on KAFKA-12426:
-

[~jolshan] Thanks, good find. Since it is not a regression, it's tough to call 
it a blocker. I'd suggest we target 2.8.1 for now, but if the patch is not too 
crazy, we might still be able to get it in depending on RC progress.

> Missing logic to create partition.metadata files in RaftReplicaManager
> --
>
> Key: KAFKA-12426
> URL: https://issues.apache.org/jira/browse/KAFKA-12426
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: Justine Olshan
>Priority: Major
>  Labels: kip-500
>
> As part of KIP-516, the broker should create a partition.metadata file for 
> each partition in order to keep track of the topicId. This is done through 
> `Partition.checkOrSetTopicId`. We have the logic for this in 
> `ReplicaManager.becomeLeaderOrFollower` already, but we need to implement 
> analogous logic in `RaftReplicaManager.handleMetadataRecords`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KAFKA-12426) Missing logic to create partition.metadata files in RaftReplicaManager

2021-03-12 Thread Justine Olshan (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-12426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17300616#comment-17300616
 ] 

Justine Olshan commented on KAFKA-12426:


While working on this bug, I caught another small bug in ReplicaManager due to 
changing the position of the topic ID check (to prevent handling a request for 
an inconsistent ID).

 I realized that on the first LISR received on a newly created topic, the log 
will not yet be created when `checkOrSetTopicId` is called. This means the log 
will not store the topic ID until the second LISR request. This bug means that 
we lose the benefits of topic IDs on this first LISR pass, but we don't 
actively break things more than LISR pre-topic IDs.

A similar event will occur when restarting brokers. Logs will not yet be 
associated with the Partition object, so we won't be able to check the topic ID 
on the first LISR pass. This means the partition.metadata file is unable to 
serve its intended purpose in this case, so I think this case is a little worse.

There is a pretty simple fix to this which I think I can include in this PR 
[~ijuma] [~vvcephei] do you think this is a blocker for 2.8?

> Missing logic to create partition.metadata files in RaftReplicaManager
> --
>
> Key: KAFKA-12426
> URL: https://issues.apache.org/jira/browse/KAFKA-12426
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: Justine Olshan
>Priority: Major
>  Labels: kip-500
>
> As part of KIP-516, the broker should create a partition.metadata file for 
> each partition in order to keep track of the topicId. This is done through 
> `Partition.checkOrSetTopicId`. We have the logic for this in 
> `ReplicaManager.becomeLeaderOrFollower` already, but we need to implement 
> analogous logic in `RaftReplicaManager.handleMetadataRecords`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)