[ https://issues.apache.org/jira/browse/KAFKA-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Grant updated KAFKA-14608: --------------------------------- Description: In KRaft mode we complete reassignments when the adding replicas have been added to the ISR - see [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.] As a result its possible for the partition to be under min ISR if the number of adding replicas is less than the topic's under min ISR config and some other target replicas are not in the ISR for whatever reason. This behavior differs to ZK mode where we require all target replicas to be in the ISR for the reassignment to complete - see [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003]. I thought more about it and I only think the reassignment can be completed *and* we are under min ISR if either 1) we were already under min ISR in the first place or 2) the replication factor decreases. So in practice I dont think this is a severe issue. Either we were already under min ISR so the reassignment does not *cause* under min ISR. Or we're decreasing the replication factor which isnt a common scenario. It seems there are two options. One is to match the ZK behavior and only complete reassignments when all target replicas are in the ISR. The second is to complete reassignments when enough target replicas are in the ISR such that we're above under min ISR. So if the under min ISR config for a topic is two, then we would complete reassignments when at least two target replicas are in the ISR. was: In KRaft mode we complete reassignments when the adding replicas have been added to the ISR - see [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.] As a result its possible for the partition to be under min ISR if the number of adding replicas is less than the topic's under min ISR config and some other target replicas are not in the ISR for whatever reason. This behavior differs to ZK mode where we require all target replicas to be in the ISR for the reassignment to complete - see [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003]. I thought more about it and I only think the reassignment can be completed *and* we are under min ISR if either 1) we were already under min ISR in the first place or 2) the replication factor decreases. So in practice I dont think this is a severe issue. It seems there are two options. One is to match the ZK behavior and only complete reassignments when all target replicas are in the ISR. The second is to complete reassignments when enough target replicas are in the ISR such that we're above under min ISR. So if the under min ISR config for a topic is two, then we would complete reassignments when at least two target replicas are in the ISR. > Look into when reassignment should be completed in KRaft mode > ------------------------------------------------------------- > > Key: KAFKA-14608 > URL: https://issues.apache.org/jira/browse/KAFKA-14608 > Project: Kafka > Issue Type: Improvement > Reporter: Andrew Grant > Assignee: Andrew Grant > Priority: Major > > In KRaft mode we complete reassignments when the adding replicas have been > added to the ISR - see > [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.] > As a result its possible for the partition to be under min ISR if the number > of adding replicas is less than the topic's under min ISR config and some > other target replicas are not in the ISR for whatever reason. > This behavior differs to ZK mode where we require all target replicas to be > in the ISR for the reassignment to complete - see > [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003]. > > I thought more about it and I only think the reassignment can be completed > *and* we are under min ISR if either 1) we were already under min ISR in the > first place or 2) the replication factor decreases. So in practice I dont > think this is a severe issue. Either we were already under min ISR so the > reassignment does not *cause* under min ISR. Or we're decreasing the > replication factor which isnt a common scenario. > It seems there are two options. One is to match the ZK behavior and only > complete reassignments when all target replicas are in the ISR. The second is > to complete reassignments when enough target replicas are in the ISR such > that we're above under min ISR. So if the under min ISR config for a topic is > two, then we would complete reassignments when at least two target replicas > are in the ISR. -- This message was sent by Atlassian Jira (v8.20.10#820010)