[jira] [Created] (KAFKA-2267) kafka-reassign-partitions.sh --generate doesn't work when reducing replication factor

[Robin Yamaguchi (JIRA)]

Robin Yamaguchi created KAFKA-2267:
--------------------------------------

             Summary: kafka-reassign-partitions.sh --generate doesn't work when 
reducing replication factor
                 Key: KAFKA-2267
                 URL: https://issues.apache.org/jira/browse/KAFKA-2267
             Project: Kafka
          Issue Type: Bug
          Components: tools
    Affects Versions: 0.8.2.1
         Environment: Centos 6.5, scala_version 2.10.4
            Reporter: Robin Yamaguchi
            Priority: Minor


Using kafka-reassign-partitions.sh --generate errors when attempting to reduce 
the replication factor of a topic.

{code}
$ /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper 
server01:2181,server02:2181,server03:2181 --topic OpsTesting

Topic:OpsTesting        PartitionCount:1        ReplicationFactor:3     Configs:
        Topic: OpsTesting       Partition: 0    Leader: 2       Replicas: 2,0,1 
Isr: 2,0,1

$ more topics-to-move.json

{"topics": [{"topic": "OpsTesting"}],
 "version":1
}

$ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper --describe 
--zookeeper server01:2181,server02:2181,server03:2181 
--topics-to-move-json-file topics-to-move.json --broker-list "1,2" --generate


Partitions reassignment failed due to replication factor: 3 larger than 
available brokers: 2
kafka.admin.AdminOperationException: replication factor: 3 larger than 
available brokers: 2
        at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
        at 
kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:97)
        at 
kafka.admin.ReassignPartitionsCommand$$anonfun$generateAssignment$1.apply(ReassignPartitionsCommand.scala:96)
        at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
        at 
kafka.admin.ReassignPartitionsCommand$.generateAssignment(ReassignPartitionsCommand.scala:96)
        at 
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:45)
        at 
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
{code}

Yet I can manually create a json file, and kafka-reassign-partitions.sh 
--execute will work:

{code}
$ more part.json

{"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[0,1]}]}

$ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper 
server01:2181,server02:2181,server03:2181 --reassignment-json-file part.json 
--execute
Current partition replica assignment

{"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[2,0,1]}]}

Save this to use as the --reassignment-json-file option during rollback
Successfully started reassignment of partitions 
{"version":1,"partitions":[{"topic":"OpsTesting","partition":0,"replicas":[0,1]}]}
{code}

kafka-reassign-partitions.sh --verify also works as expected after the 
--execute:

{code}
$ /usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper 
server01:2181,server02:2181,server03:2181 --topic OpsTesting
Topic:OpsTesting        PartitionCount:1        ReplicationFactor:2     Configs:
        Topic: OpsTesting       Partition: 0    Leader: 0       Replicas: 0,1   
Isr: 0,1


$ /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper 
server01:2181,server02:2181,server03:2181 --reassignment-json-file part.json 
--verify
Status of partition reassignment:
Reassignment of partition [OpsTesting,0] completed successfully
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)