Hi, We have several Kafka clusters in production, and we've had to reassign replication a few times now in production. Some of our topic/partitions are pretty large, up to 32 partitions per topic, and 16GB per partition, so adding a new broker and/or repairing a broker that had been down for some time turns out to be a major undertaking.
Today, when we attempt to replicate a single partition, it pegs the disk IO, and uses a significant chunk of the 10Gbps interface for a good ~5 minutes. This is causing problems for our downstream consumers, which rely on having a consistent stream of realtime data being sent to them. Is there a way to throttle Kafka replication between nodes, so that instead of it going full blast, it will replicate at a fixed rate in megabytes or activities/batches per second? Or maybe is this planned for a future release, maybe 0.9? Thanks, Marcos Juarez
