[
https://issues.apache.org/jira/browse/KAFKA-2106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14710364#comment-14710364
]
Allen Wang commented on KAFKA-2106:
-----------------------------------
I am curious in the following code why there is a restriction that partitions
should have no more than 3 replicas and every partition has the same number or
replicas. It is possible that different topic has different number of replicas
given different requirement of availability.
{code}
def filterValidTopicAssignment() = {
val groupedByTopic = allTopicsAssignment.groupBy(tp => tp._1.topic)
/**
* check replicas:
* replicas amount should be more than 0 and less than 3
* all partitions should have the same amount of replicas
*
*/
var validTopicAssignment = groupedByTopic.filter(
t => {
t._2.head._2.size > 0 && t._2.head._2.size < 3 && t._2.values.map(seq
=> seq.length).toSet.size == 1
}
)
if(includeTopicSet.size != 0) {
validTopicAssignment = validTopicAssignment.filter(topicInfo =>
includeTopicSet.contains(topicInfo._1))
}
if(excludeTopicSet.size != 0) {
validTopicAssignment = validTopicAssignment.filter(topicInfo => (!
excludeTopicSet.contains(topicInfo._1)))
}
validTopicAssignment
}
{code}
> Partition balance tool between borkers
> --------------------------------------
>
> Key: KAFKA-2106
> URL: https://issues.apache.org/jira/browse/KAFKA-2106
> Project: Kafka
> Issue Type: New Feature
> Components: admin
> Affects Versions: 0.8.3
> Reporter: chenshangan
> Attachments: KAFKA-2106.3, KAFKA-2106.patch, KAFKA-2106.patch.2
>
>
> The default partition assignment algorithm can work well in a static kafka
> cluster(number of brokers seldom change). Actually, in production env, number
> of brokers is always increasing according to the business data. When new
> brokers added to the cluster, it's better to provide a tool that can help to
> move existing data to new brokers. Currently, users need to choose topic or
> partitions manually and use the Reassign Partitions Tool
> (kafka-reassign-partitions.sh) to achieve the goal. It's a time-consuming
> task when there's a lot of topics in the cluster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
