Hi Akash Jain Thanks for the reply seeking help for the same to choose hashing logics. Please refer/suggest any.
On Sat, Aug 17, 2024 at 10:21 AM Akash Jain <akashjain0...@gmail.com> wrote: > Hi Karthick. You could implement your own custom partitioner. > > On Saturday, August 17, 2024, Karthick <ibmkarthickma...@gmail.com> wrote: > > > Hi Team, > > > > I'm using Kafka partitioning to maintain field-based ordering across > > partitions, but I'm experiencing data skewness among the partitions. I > have > > 96 partitions, and I'm sending data with 500 distinct keys that are used > > for partitioning. While monitoring the Kafka cluster, I noticed that a > few > > partitions are underutilized while others are overutilized. > > > > This seems to be a hashing problem. Can anyone suggest a better hashing > > technique or partitioning strategy to balance the load more effectively? > > > > Thanks in advance for your help. > > >
