Hi I have been trying a small experiment with partition reassignment in cloud. where instead of copying data between brokers using network, I moved the disk between the 2 brokers and ran the partition reassignment. This actually increased the speed of partition reassignment significantly. (As it had to catchup/fetch only down time data)
I tried this experiment with Kafka 2.2.1 and it worked. I validated the data-consistency using "kafka-replica-verification.sh" script and also by comparing md5 hash of the log and index files. Few more details of the experiment: * Both the brokers from and to which the partitions are moving had to be shutdown. * All the partitions in the disk are moved at once to new broker. * Had to update broker.id property in meta.properties file for the moved log directory before broker restart . * Had to re-balance Leaders after brokers restart. Can you please let me know if this approach will work in production ? Is there any scenario where it might truncate/delete all log files in moved disk and fetch the complete data from the leader partition ? Thanks Varun
