Hello Rajat, I'm not sure what you mean to "reshuffle messages", because once the messages are stored in brokers, they can't be modified anymore. But if you want to make the previous added messages route to the same partitions after partition increasing, you can write custom partitioner: https://kafka.apache.org/documentation/#producerconfigs_partitioner.class
So, for example, you added 10 partitions for some new keys (ex: key-101 ~ key-110), you can write the partitioner to route the old keys to [hash(key) % 100(old partition count)], and new keys route to [hash(key) % 10(new added partitions)]. Thank you. Luke On Sun, Nov 21, 2021 at 7:46 PM rajat kumar <kumar.rajat20...@gmail.com> wrote: > Hello Users, > > I am pretty new to Kafka, we will have key based messages coming up in > kafka. > We will have a 5 node cluster and I am going ahead with a 100 partition for > the topic for now. > Let's say if there is a need to increase the number of partitions. How do I > reshuffle messages , since previously added messages would end up in the > wrong partition as per hash partition algo? > > Thanks > Rajat >
