[
https://issues.apache.org/jira/browse/KAFKA-14608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Grant updated KAFKA-14608:
---------------------------------
Description:
In KRaft mode we complete reassignments when the adding replicas have been
added to the ISR - see
[https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.]
As a result its possible for the partition to be under min ISR if the number
of adding replicas is less than the topic's under min ISR config and some other
target replicas are not in the ISR for whatever reason.
This behavior differs to ZK mode where we require all target replicas to be in
the ISR for the reassignment to complete - see
[https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003].
I thought more about it and I only think the reassignment can be completed
*and* we are under min ISR if either 1) we were already under min ISR in the
first place or 2) the replication factor decreases. So in practice I dont think
this is a severe issue.
It seems there are two options. One is to match the ZK behavior and only
complete reassignments when all target replicas are in the ISR. The second is
to complete reassignments when enough target replicas are in the ISR such that
we're above under min ISR. So if the under min ISR config for a topic is two,
then we would complete reassignments when at least two target replicas are in
the ISR.
was:
In KRaft mode we complete reassignments when the adding replicas have been
added to the ISR - see
[https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.]
As a result its possible for the partition to be under min ISR if the number
of adding replicas is less than the topic's under min ISR config and some other
target replicas are not in the ISR for whatever reason.
This behavior differs to ZK mode where we require all target replicas to be in
the ISR for the reassignment to complete - see
[https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003].
It seems there are two options. One is to match the ZK behavior and only
complete reassignments when all target replicas are in the ISR. The second is
to complete reassignments when enough target replicas are in the ISR such that
we're above under min ISR. So if the under min ISR config for a topic is two,
then we would complete reassignments when at least two target replicas are in
the ISR.
> Make sure reassignment does not cause under min ISR
> ---------------------------------------------------
>
> Key: KAFKA-14608
> URL: https://issues.apache.org/jira/browse/KAFKA-14608
> Project: Kafka
> Issue Type: Improvement
> Reporter: Andrew Grant
> Assignee: Andrew Grant
> Priority: Major
>
> In KRaft mode we complete reassignments when the adding replicas have been
> added to the ISR - see
> [https://github.com/apache/kafka/blob/trunk/metadata/src/main/java/org/apache/kafka/controller/PartitionChangeBuilder.java#L288.]
> As a result its possible for the partition to be under min ISR if the number
> of adding replicas is less than the topic's under min ISR config and some
> other target replicas are not in the ISR for whatever reason.
> This behavior differs to ZK mode where we require all target replicas to be
> in the ISR for the reassignment to complete - see
> [https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala#L1003].
>
> I thought more about it and I only think the reassignment can be completed
> *and* we are under min ISR if either 1) we were already under min ISR in the
> first place or 2) the replication factor decreases. So in practice I dont
> think this is a severe issue.
> It seems there are two options. One is to match the ZK behavior and only
> complete reassignments when all target replicas are in the ISR. The second is
> to complete reassignments when enough target replicas are in the ISR such
> that we're above under min ISR. So if the under min ISR config for a topic is
> two, then we would complete reassignments when at least two target replicas
> are in the ISR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
