Hello, here is the update .. when i ran script - kafka-preferred-replica-election, it did the re-election as required.
./bin/kafka-preferred-replica-election --zookeeper localhost:3181 so does that mean that i need to run the script -> ./bin/confluent-rebalancer to rebalance the data, but for the leader election, the script to be run is -> ./bin/kafka-preferred-replica-election The documentation mentions this (link - http://docs.confluent.io/current/kafka/rebalancer/rebalancer.html) The confluent-rebalancer tool balances data so that the number of leaders > and disk usage are even across brokers and racks on a per topic and cluster > level while minimising data movement. seems there is a disconnect here, pls let me know if anyone has inputs. On Tue, Aug 1, 2017 at 4:45 PM, karan alang <karan.al...@gmail.com> wrote: > Hi All - > i'm trying to rebalance Kafka topic (refer link -> > http://docs.confluent.io/current/kafka/rebalancer/rebalancer.html), and > somehow the rebalancing is not working. > > > Here is what i'm doing -> > - i've 4 Kafka brokers & i've made changes to the server.properties file > to enable Confluent Metrics Reporter. > (attached are the server.properties of the 4 brokers) > > -> Created a topic specifying Replica assignment > > ./bin/kafka-topics --create --topic topic-a1 --replica-assignment > 0:1,0:1,0:1,0:1 --zookeeper nwk2-bdp-kafka-04.gdcs-qa.apple.com:3181 > > -> describe topic > > ./bin/kafka-topics --describe --topic topic-a1 --zookeeper >> nwk2-bdp-kafka-04.gdcs-qa.apple.com:3181 >> Topic:topic-a1 PartitionCount:4 ReplicationFactor:2 Configs: >> Topic: topic-a1 Partition: 0 Leader: 1 Replicas: 1,0 Isr: 0,1 >> Topic: topic-a1 Partition: 1 Leader: 1 Replicas: 1,0 Isr: 0,1 >> Topic: topic-a1 Partition: 2 Leader: 0 Replicas: 0,1 Isr: 0,1 >> Topic: topic-a1 Partition: 3 Leader: 0 Replicas: 0,1 Isr: 0,1 >> > > > -> Produce data into topics, using the following command > > ./bin/kafka-producer-perf-test --topic topic-a1 --num-records 200000 >> --record-size 1000 --throughput 10000000 --producer-props bootstrap.servers= >> nwk2-bdp-kafka-04.gdcs-qa.apple.com:9092,nwk2-bdp-kafka-04.gdcs- >> qa.apple.com:9082,nwk2-bdp-kafka-04.gdcs-qa.apple.com:9072, >> nwk2-bdp-kafka-04.gdcs-qa.apple.com:9062 > > > > -> Force Creation of offsets topic, by creating a Consumer (NOT SURE WHAT > THIS IS FOR ???) : > > ./bin/kafka-consumer-perf-test --topic topic-a1 --broker-list >> nwk2-bdp-kafka-04.gdcs-qa.apple.com:9092,nwk2-bdp-kafka- >> 04.gdcs-qa.apple.com:9082,nwk2-bdp-kafka-04.gdcs-qa.apple.com:9072, >> nwk2-bdp-kafka-04.gdcs-qa.apple.com:9062 --messages 10 > > > > -> run the following command to rebalance > >> >> The plan that is presented does not really any rebalancing -> > > ./bin/confluent-rebalancer execute --zookeeper nwk2-bdp-kafka-04.gdcs-qa. >> apple.com:3181 --metrics-bootstrap-server nwk2-bdp-kafka-04.gdcs-qa. >> apple.com:9092,nwk2-bdp-kafka-04.gdcs-qa.apple.com:9082,nwk2 >> -bdp-kafka-04.gdcs-qa.apple.com:9072,nwk2-bdp-kafka-04. >> gdcs-qa.apple.com:9062 --throttle 10000000 --verbose >> >> Computing the rebalance plan (this may take a while) ... >> You are about to move 0 replica(s) for 0 partitions to 0 broker(s) with >> total size 0 MB. >> The preferred leader for 2 partition(s) will be changed. >> In total, the assignment for 2 partitions will be changed. >> The minimum free volume space is set to 20.0%. >> The following brokers will have less than 40% of free volume space during >> the rebalance: >> Broker Current Size (MB) Size During Rebalance (MB) Free % During >> Rebalance Size After Rebalance (MB) Free % After Rebalance >> 0 4,021.1 4,021.1 14.2 >> 4,021.1 14.2 >> 1 1,240.8 1,240.8 14.2 >> 1,240.8 14.2 >> 2 620.4 620.4 14.2 >> 620.4 14.2 >> 3 0 0 14.2 >> 0 14.2 >> Min/max stats for brokers (before -> after): >> Type Leader Count Replica Count Size >> (MB) >> Min 0 (id: 3) -> 0 (id: 3) 0 (id: 3) -> 0 (id: 3) 0 (id: >> 3) -> 0 (id: 3) >> Max 125 (id: 0) -> 123 (id: 0) 127 (id: 0) -> 127 (id: 0) 4,021.1 >> (id: 0) -> 4,021.1 (id: 0) >> No racks are defined. >> Broker stats (before -> after): >> Broker Leader Count Replica Count Size (MB) Free >> Space (%) >> 0 125 -> 123 127 -> 127 4,021.1 -> 4,021.1 14.2 -> >> 14.2 >> 1 3 -> 5 12 -> 12 1,240.8 -> 1,240.8 14.2 -> >> 14.2 >> 2 2 -> 2 3 -> 3 620.4 -> 620.4 14.2 -> >> 14.2 >> 3 0 -> 0 0 -> 0 0 -> 0 14.2 -> >> 14.2 >> Would you like to continue? (y/n): y >> The rebalance has been started, run `status` to check progress. >> Warning: You must run the `status` or `finish` command periodically, >> until the rebalance completes, to ensure the throttle is removed. You can >> also alter the throttle by re-running the execute command passing a new >> value. >> >> > > -> I describe the topic, after rebalancing > > ./bin/kafka-topics --describe --topic topic-a1 --zookeeper >> nwk2-bdp-kafka-04.gdcs-qa.apple.com:3181 >> Topic:topic-a1 PartitionCount:4 ReplicationFactor:2 Configs: >> Topic: topic-a1 Partition: 0 Leader: 1 Replicas: 1,0 Isr: 0,1 >> Topic: topic-a1 Partition: 1 Leader: 1 Replicas: 1,0 Isr: 0,1 >> Topic: topic-a1 Partition: 2 Leader: 0 Replicas: 0,1 Isr: 0,1 >> Topic: topic-a1 Partition: 3 Leader: 0 Replicas: 0,1 Isr: 0,1 >> >> > > My expectation was that the topic will have all 4 brokers as leaders for > the 4 partitions, but that does not seem to be happening. > > Any ideas what the issue is ? > >
