Re: Keys and partitions

2020-07-07 Thread Ricardo Ferreira
It is also important to note that since the release 2.4 of Apache Kafka the DefaultPartitioner now implements a sticky partitioning strategy rather than round-robin based on the key. This means that if you need fine control over which partition records will end up given the key -- you ought to

Re: Keys and partitions

2020-07-07 Thread Vinicius Scheidegger
Hi Victoria, If processing order is not a requirement you could define a random key and your load would be randomly distributed across partitions. So far I was unable to find a solution to perfectly distribute the load across partitions when records are created from multiple producers - random

Keys and partitions

2020-07-06 Thread Victoria Zuberman
Hi, I have userId as a key. Many users have moderate amounts of data but some users have more and some users have huge amount of data. I have been thinking about the following aspects of partitioning: 1. If two or more large users will fall into same partition I might end up with large