[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Priority: Major (was: Minor) > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Major > Labels: kip-500 > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > and I think MetadataPartition should include topic uuid, and the topic name > is optional > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > but the MetadataPartition doesn't include topic uuid, it only includes topic > name, when to process, it can't find the origin topic uuid, and find the > latest the topic id, but it's not right. and delete topic t1 should do before > create t1 or change p1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Gustafson updated KAFKA-12682: Labels: kip-500 (was: ) > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > Labels: kip-500 > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > and I think MetadataPartition should include topic uuid, and the topic name > is optional > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > but the MetadataPartition doesn't include topic uuid, it only includes topic > name, when to process, it can't find the origin topic uuid, and find the > latest the topic id, but it's not right. and delete topic t1 should do before > create t1 or change p1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only includes topic name, when to process, it can't find the origin topic uuid, and find the latest the topic id, but it's not right. and delete topic t1 should do before create t1 or change p1. was: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional, the topic name only can find the partition by the for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only include topic name, when to stop, it can't find the origin topic id, and find the latest the topic id, but it's not right. > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > and I think MetadataPartition should include topic uuid, and the topic name > is optional > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > but the MetadataPartition doesn't include topic uuid, it only includes topic > name, when to process, it can't find the origin topic uuid, and find the > latest the topic id, but it's not right. and delete topic t1 should do before > create t1 or change p1. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional, the topic name only can find the partition by the for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only include topic name, when to stop, it can't find the origin topic id, and find the latest the topic id, but it's not right. was: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional, the topic name only can find the partition by the for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only include topic name, when to stop, it can't find the origin topic id, and find the latest the topic id, but it's not right. > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > and I think MetadataPartition should include topic uuid, and the topic name > is optional, the topic name only can find the partition by the > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > but the MetadataPartition doesn't include topic uuid, it only include topic > name, when to stop, it can't find the origin topic id, and find the latest > the topic id, but it's not right. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional, the topic name only can find the partition by the for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only include topic name, when to stop, it can't find the origin topic id, and find the latest the topic id, but it's not right. was: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional。 for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only include topic name, when to stop, it can't find the origin topic id, and find the latest the topic id, but it's not right. > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > and I think MetadataPartition should include topic uuid, and the topic name > is optional, the topic name only can find the partition by the > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > create topic t1, delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > but the MetadataPartition doesn't include topic uuid, it only include topic > name, when to stop, it can't find the origin topic id, and find the latest > the topic id, but it's not right. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. and I think MetadataPartition should include topic uuid, and the topic name is optional。 for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 but the MetadataPartition doesn't include topic uuid, it only include topic name, when to stop, it can't find the origin topic id, and find the latest the topic id, but it's not right. was: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. I think handleMetadataRecords process batch records, and the records need duplicate removal group by partition and operation(change(create ，update), delete)。 for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 by the way, I think MetadataPartition should include the topic uuid, not only topic name. > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > and I think MetadataPartition should include topic uuid, and the topic name > is optional。 > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > create topic t1, delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > but the MetadataPartition doesn't include topic uuid, it only include topic > name, when to stop, it can't find the origin topic id, and find the latest > the topic id, but it's not right. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. I think handleMetadataRecords process batch records, and the records need duplicate removal group by partition and operation(change(create ，update), delete)。 for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 by the way, I think MetadataPartition should include the topic uuid, not only topic name. was: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. I think handleMetadataRecords process batch records, and the records need duplicate removal group by partition and operation(change(create ，update), delete)。 for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > I think handleMetadataRecords process batch records, and the records need > duplicate removal group by partition and operation(change(create ，update), > delete)。 > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > create topic t1, delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 > by the way, I think MetadataPartition should include the topic uuid, not only > topic name. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (KAFKA-12682) Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order
[ https://issues.apache.org/jira/browse/KAFKA-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jacky updated KAFKA-12682: -- Description: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. I think handleMetadataRecords process batch records, and the records need duplicate removal group by partition and operation(change(create ，update), delete)。 for example, create topic t1, delete topic t1, create topic t1, change leader of p1 and then compact the records create topic t1, delete topic t1, change t1, p1 but currently, implementation will be 1. process change t1, p1 2. process delete topic t1 was: In version 2.8, MetadataPartitionsBuilder has the field _localChanged and _localRemoved which record the change and delete partition, but we always process _localChanged partitions, and then _localRemoved in the kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the original order, for example, 1. migrate the partition p1 from b0 to b1; 2. change the leader of p1 3.migrate p1 from b1 to b0 and the _localRemoved will delete the p1 at last. > Kraft MetadataPartitionsBuilder _localChanged and _localRemoved out of order > - > > Key: KAFKA-12682 > URL: https://issues.apache.org/jira/browse/KAFKA-12682 > Project: Kafka > Issue Type: Bug > Components: core >Affects Versions: 2.8.0 >Reporter: jacky >Priority: Minor > > In version 2.8, MetadataPartitionsBuilder has the field _localChanged and > _localRemoved which record the change and delete partition, but we always > process _localChanged partitions, and then _localRemoved in the > kafka.server.RaftReplicaManager#handleMetadataRecords, not respect the > original order, for example, > 1. migrate the partition p1 from b0 to b1; > 2. change the leader of p1 > 3.migrate p1 from b1 to b0 > and the _localRemoved will delete the p1 at last. > I think handleMetadataRecords process batch records, and the records need > duplicate removal group by partition and operation(change(create ，update), > delete)。 > for example, > create topic t1, delete topic t1, create topic t1, change leader of p1 > and then compact the records > create topic t1, delete topic t1, change t1, p1 > but currently, implementation will be > 1. process change t1, p1 > 2. process delete topic t1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
