Re: Handling load distribution evenly, with ordering

2024-08-16 Thread Akash Jain
Hi Karthick. You could implement your own custom partitioner. On Saturday, August 17, 2024, Karthick wrote: > Hi Team, > > I'm using Kafka partitioning to maintain field-based ordering across > partitions, but I'm experiencing data skewness among the partitions. I have > 96 partitions, and I'm s

Handling load distribution evenly, with ordering

2024-08-16 Thread Karthick
Hi Team, I'm using Kafka partitioning to maintain field-based ordering across partitions, but I'm experiencing data skewness among the partitions. I have 96 partitions, and I'm sending data with 500 distinct keys that are used for partitioning. While monitoring the Kafka cluster, I noticed that a