[
https://issues.apache.org/jira/browse/KAFKA-1464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371402#comment-15371402
]
K Zakee commented on KAFKA-1464:
--------------------------------
I agree with Ralph.
Lets say, we have a high produce rate and a system failure (as long as the
kafka retention period itself), there is a lot of data to catchup and as fast
as it could. Throttling catching up of out-of-sync replicas in this case may
become a "chase-your-own-tail thing" and these may never be able to catchup
with their leader or take days depending on produce-rate and throttle limit.
Suppressing new replicas taking the leadership until the time they have all
caught up sounds a better idea.
> Add a throttling option to the Kafka replication tool
> -----------------------------------------------------
>
> Key: KAFKA-1464
> URL: https://issues.apache.org/jira/browse/KAFKA-1464
> Project: Kafka
> Issue Type: New Feature
> Components: replication
> Affects Versions: 0.8.0
> Reporter: mjuarez
> Assignee: Ben Stopford
> Priority: Minor
> Labels: replication, replication-tools
> Fix For: 0.10.1.0
>
>
> When performing replication on new nodes of a Kafka cluster, the replication
> process will use all available resources to replicate as fast as possible.
> This causes performance issues (mostly disk IO and sometimes network
> bandwidth) when doing this in a production environment, in which you're
> trying to serve downstream applications, at the same time you're performing
> maintenance on the Kafka cluster.
> An option to throttle the replication to a specific rate (in either MB/s or
> activities/second) would help production systems to better handle maintenance
> tasks while still serving downstream applications.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
