Hi Mich,
You might want to take a look at this section: "Increasing replication
factor" in the doc:
https://kafka.apache.org/documentation/#basic_ops_increase_replication_factor
Simply put, the json file provided in kafka-reassign-partitions.sh should
show the final replicas assignment after this operation.
In your case, the size of each replica should be 3, but I saw you put 9
replicas there.
Hope it helps.
Luke
On Sat, May 13, 2023 at 7:20 PM Mich Talebzadeh <mich.talebza...@gmail.com>
wrote:
> Hi,
>
> From the following list
>
> kafka-topics.sh --describe --bootstrap-server rhes75:9092 --topic md
>
> Topic: md TopicId: UfQly87bQPCbVKoH-PQheg PartitionCount: 9
> ReplicationFactor: 9 Configs:
> segment.bytes=1073741824,retention.bytes=1073741824
> Topic: md Partition: 0 Leader: 12 Replicas:
> 12,10,8,2,9,11,1,7,3 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 1 Leader: 9 Replicas:
> 9,8,2,12,11,1,7,3,10 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 2 Leader: 11 Replicas:
> 11,2,12,9,1,7,3,10,8 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 3 Leader: 1 Replicas:
> 1,12,9,11,7,3,10,8,2 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 4 Leader: 7 Replicas:
> 7,9,11,1,3,10,8,2,12 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 5 Leader: 3 Replicas:
> 3,11,1,7,10,8,2,12,9 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 6 Leader: 10 Replicas:
> 10,1,7,3,8,2,12,9,11 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 7 Leader: 8 Replicas:
> 8,7,3,10,2,12,9,11,1 Isr: 10,1,9,2,12,7,3,11,8
> Topic: md Partition: 8 Leader: 2 Replicas:
> 2,3,10,8,12,9,11,1,7 Isr: 10,1,9,2,12,7,3,11,8
>
> so for topic md I have 9 Partitions and 9 Replication
>
> As for redundancy and prevent data loss I only need 3 replicas (the leader
> and 2 followers) , so I use the following to reduce the number of replicas
> to 3
>
>
> {
> "version":1,
> "partitions":[
> {"topic":"md","partition":0,"replicas":[12,10,8,2,9,11,1,7,3]},
> {"topic":"md","partition":1,"replicas":[9,8,2,12,11,1,7,3,10]},
> {"topic":"md","partition":2,"replicas":[11,2,12,9,1,7,3,10,8]}
> ]
> }
> with the following command
>
> kafka-reassign-partitions.sh --bootstrap-server rhes75:9092
> --reassignment-json-file ./reduce_replication_factor2.json --execute
>
> and this is the output
>
> Current partition replica assignment
>
>
> {"version":1,"partitions":[{"topic":"md","partition":0,"replicas":[12,10,8,2,9,11,1,7,3],"log_dirs":["any","any","any","any","any","any","any","any","any"]},{"topic":"md","partition":1,"replicas":[9,8,2,12,11,1,7,3,10],"log_dirs":["any","any","any","any","any","any","any","any","any"]},{"topic":"md","partition":2,"replicas":[11,2,12,9,1,7,3,10,8],"log_dirs":["any","any","any","any","any","any","any","any","any"]}]}
>
> Save this to use as the --reassignment-json-file option during rollback
>
> *Successfully started partition reassignments for md-0,md-1,md-2*
>
>
> It says it is doing it, but nothing is happening!
>
>
> This is the size of Kafka Topic in MB per each per partition remaining:
>
>
> kafka-log-dirs.sh --bootstrap-server rhes75:9092 --topic-list md --describe
> | grep -oP '(?<=size":)\d+' | awk '{ sum += $1 } END { print
> sum/1024/1024/9 }'
>
>
> Which comes back with 81.5 MB
>
>
> Will this work as I have stopped the queue but still data there. In short,
> is downsizing practical?
>
>
> Thanks
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
