[
https://issues.apache.org/jira/browse/KAFKA-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Grant updated KAFKA-14608:
---------------------------------
Description:
In KRaft mode we complete reassignments when the adding replicas have been
added to the ISR - see
[https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.]
As a result its possible for the partition to be under min ISR if the number
of adding replicas is less than the topic's under min ISR config and some other
target replicas are not in the ISR for whatever reason.
This behavior differs to ZK mode where we require all target replicas to be in
the ISR for the reassignment to complete - see
[https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003].
I thought more about it and I only think the reassignment can be completed
*and* we get to be under min ISR if either 1) we were already under min ISR in
the first place or 2) the replication factor decreases. So in practice I don't
think this is a severe issue. Either we were already under min ISR so the
reassignment does not actually *cause* under min ISR. Or we're decreasing the
replication factor which isn't a common scenario.
It seems there are two options. One is to match the ZK behavior and only
complete reassignments when all target replicas are in the ISR. The second is
to complete reassignments when enough target replicas are in the ISR such that
we're above under min ISR. So if the under min ISR config for a topic is two,
then we would complete reassignments when at least two target replicas are in
the ISR.
was:
In KRaft mode we complete reassignments when the adding replicas have been
added to the ISR - see
[https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.]
As a result its possible for the partition to be under min ISR if the number
of adding replicas is less than the topic's under min ISR config and some other
target replicas are not in the ISR for whatever reason.
This behavior differs to ZK mode where we require all target replicas to be in
the ISR for the reassignment to complete - see
[https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003].
I thought more about it and I only think the reassignment can be completed
*and* we are under min ISR if either 1) we were already under min ISR in the
first place or 2) the replication factor decreases. So in practice I dont think
this is a severe issue. Either we were already under min ISR so the
reassignment does not *cause* under min ISR. Or we're decreasing the
replication factor which isnt a common scenario.
It seems there are two options. One is to match the ZK behavior and only
complete reassignments when all target replicas are in the ISR. The second is
to complete reassignments when enough target replicas are in the ISR such that
we're above under min ISR. So if the under min ISR config for a topic is two,
then we would complete reassignments when at least two target replicas are in
the ISR.
> Look into when reassignment should be completed in KRaft mode
> -------------------------------------------------------------
>
> Key: KAFKA-14608
> URL: https://issues.apache.org/jira/browse/KAFKA-14608
> Project: Kafka
> Issue Type: Improvement
> Reporter: Andrew Grant
> Assignee: Andrew Grant
> Priority: Major
>
> In KRaft mode we complete reassignments when the adding replicas have been
> added to the ISR - see
> [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.]
> As a result its possible for the partition to be under min ISR if the number
> of adding replicas is less than the topic's under min ISR config and some
> other target replicas are not in the ISR for whatever reason.
> This behavior differs to ZK mode where we require all target replicas to be
> in the ISR for the reassignment to complete - see
> [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003].
>
> I thought more about it and I only think the reassignment can be completed
> *and* we get to be under min ISR if either 1) we were already under min ISR
> in the first place or 2) the replication factor decreases. So in practice I
> don't think this is a severe issue. Either we were already under min ISR so
> the reassignment does not actually *cause* under min ISR. Or we're decreasing
> the replication factor which isn't a common scenario.
> It seems there are two options. One is to match the ZK behavior and only
> complete reassignments when all target replicas are in the ISR. The second is
> to complete reassignments when enough target replicas are in the ISR such
> that we're above under min ISR. So if the under min ISR config for a topic is
> two, then we would complete reassignments when at least two target replicas
> are in the ISR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
