Hi Karthick. You could implement your own custom partitioner. On Saturday, August 17, 2024, Karthick <ibmkarthickma...@gmail.com> wrote:
> Hi Team, > > I'm using Kafka partitioning to maintain field-based ordering across > partitions, but I'm experiencing data skewness among the partitions. I have > 96 partitions, and I'm sending data with 500 distinct keys that are used > for partitioning. While monitoring the Kafka cluster, I noticed that a few > partitions are underutilized while others are overutilized. > > This seems to be a hashing problem. Can anyone suggest a better hashing > technique or partitioning strategy to balance the load more effectively? > > Thanks in advance for your help. >
