HanXu created KAFKA-16583: ----------------------------- Summary: Update from 3.4.0 to 3.7.0 image write failed in Kraft mode Key: KAFKA-16583 URL: https://issues.apache.org/jira/browse/KAFKA-16583 Project: Kafka Issue Type: Bug Components: kraft Affects Versions: 3.7.0 Reporter: HanXu Assignee: HanXu
How to reproduce: 1. Launch a 3.4.0 controller and a 3.4.0 broker(BrokerA) in Kraft mode; 2. Create a topic with 1 partition; 3. Launch a 3.4.0 broker(Broker B) in Kraft mode and reassign the step 2 partition to Broker B; 4. Upgrade Broker B to 3.7.0; The Broker B will keep log the following error: {code:java} [2024-04-18 14:46:54,144] ERROR Encountered metadata loading fault: Unhandled error initializing new publishers (org.apache.kafka.server.fault.LoggingFaultHandler) org.apache.kafka.image.writer.UnwritableMetadataException: Metadata has been lost because the following could not be represented in metadata version 3.4-IV0: the directory assignment state of one or more replicas at org.apache.kafka.image.writer.ImageWriterOptions.handleLoss(ImageWriterOptions.java:94) at org.apache.kafka.metadata.PartitionRegistration.toRecord(PartitionRegistration.java:391) at org.apache.kafka.image.TopicImage.write(TopicImage.java:71) at org.apache.kafka.image.TopicsImage.write(TopicsImage.java:84) at org.apache.kafka.image.MetadataImage.write(MetadataImage.java:155) at org.apache.kafka.image.loader.MetadataLoader.initializeNewPublishers(MetadataLoader.java:295) at org.apache.kafka.image.loader.MetadataLoader.lambda$scheduleInitializeNewPublishers$0(MetadataLoader.java:266) at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:127) at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:210) at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:181) at java.base/java.lang.Thread.run(Thread.java:840) {code} Bug: - When reassigning partition, PartitionRegistration#merge will set the new replicas with UNASSIGNED directory; - But in metadata version 3.4.0 PartitionRegistration#toRecord only allows MIGRATING directory; {code:java} if (options.metadataVersion().isDirectoryAssignmentSupported()) { record.setDirectories(Uuid.toList(directories)); } else { for (Uuid directory : directories) { if (!DirectoryId.MIGRATING.equals(directory)) { options.handleLoss("the directory assignment state of one or more replicas"); break; } } } {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)