Short answer - you cannot. The existing data is not reprocessed since kafka itself has no knowledge on how you did your partitioning.
The normal workaround is that you stop producers and consumers. Create a new topic with the desired number of partitions. Consume the old topic from beginning and write all data to new topic. Restart producers and consumers from your new topic. You most likely will mess up your consumer offsets. 2017-08-21 8:32 GMT+02:00 Sachin Mittal <sjmit...@gmail.com>: > Hi, > I have a topic which has four partitions and data is distributed among > those based on a specified key. > > If I want to increase the number of partitions to six how can I do the same > and also making sure that messages for a given key always go to one > (specific) partition only. > > Will the existing message redistribute themselves among new partition. > > Also say earlier message of key A went to partition 1 and going forward any > new message go to same partition where earlier messages for that key are? > > And by increasing partitions some keys may use a different partition now, > so how do I ensure the case of all messages of that key belong to single > partition. > > Thanks > Sachin >
