[
https://issues.apache.org/jira/browse/KAFKA-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ben Stopford updated KAFKA-4216:
--------------------------------
Description:
The current algorithm in kafka-reassign-partitions applies a throttle to all
moving replicas, be they leader-side or follower-side.
A preferable solution would be to change the throttled replica list to specify
whether the throttle applies to leader or follower. That way we can ensure that
the regular replication will not be throttled.
To do this we should change the way the throttled replica list is specified so
it is spread over two separate properties. One that corresponds to the
leader-side throttle, and the other that corresponds to the follower-side
throttle.
quota.leader.replication.throttled.replicas =
[partId]:[replica],[partId]:[replica],[partId]:[replica]
quota.follower.replication.throttled.replicas =
[partId]:[replica],[partId]:[replica],[partId]:[replica]
Then, when applying the throttle, the leader quota can be applied to all
current replicas, and the follower quota can be applied only to the new
replicas we are creating as part of the rebalance.
was:
The current algorithm in kafka-reassign-partitions applies a throttle to all
moving replicas, be they leader-side or follower-side.
The problem is that the leader may not be moving, which would mean it would not
have a throttle applied. So the throttle should be applied to:
[all existing replicas] ++ [all proposed replicas that moved]
A preferable solution would be to change the throttled replica list to specify
whether the throttle applies to leader or follower. That way we can ensure that
the regular replication will not be throttled.
> Replication Throttling: Leader may not be throttled if it is not "moving"
> -------------------------------------------------------------------------
>
> Key: KAFKA-4216
> URL: https://issues.apache.org/jira/browse/KAFKA-4216
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.10.1.0
> Reporter: Ben Stopford
> Assignee: Ben Stopford
>
> The current algorithm in kafka-reassign-partitions applies a throttle to all
> moving replicas, be they leader-side or follower-side.
> A preferable solution would be to change the throttled replica list to
> specify whether the throttle applies to leader or follower. That way we can
> ensure that the regular replication will not be throttled.
> To do this we should change the way the throttled replica list is specified
> so it is spread over two separate properties. One that corresponds to the
> leader-side throttle, and the other that corresponds to the follower-side
> throttle.
> quota.leader.replication.throttled.replicas =
> [partId]:[replica],[partId]:[replica],[partId]:[replica]
> quota.follower.replication.throttled.replicas =
> [partId]:[replica],[partId]:[replica],[partId]:[replica]
> Then, when applying the throttle, the leader quota can be applied to all
> current replicas, and the follower quota can be applied only to the new
> replicas we are creating as part of the rebalance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
