Murmur hash is an option. I believe it was used on Storm's v1 codebase On Sat, Aug 17, 2024, 13:57 Karthick <ibmkarthickma...@gmail.com> wrote:
> Thanks Sahil, Aaron for the reply > > Are you encountering a performance problem? > > Yes facing slowness on the executors with more load. > > Is the fields grouping absolutely required for your use case? > > Yes to maintain the ordering it required for my case. > > Can you suggest some hashes? > > > Does your case needs packets of same hash to go to same executor or >> ordering of events ? >> > > Yes, ordering of events is important. > > Currently we are using a unique id which datatype is bigint(Long). > > Can you suggest any changes reg this. > > > On Sat, Aug 17, 2024 at 8:40 AM Sahil Kamboj <sahilkamboj...@gmail.com> > wrote: > >> Hey karthick >> >> Does your case needs packets of same hash to go to same executor or >> ordering of events ? If not then go with local or shuffle grouping which >> will distribute load in round robin manner across executors. Otherwise >> choose your key wisely to avoid skewness when using fields grouping. >> >> >> >> On Sat, 17 Aug 2024 at 5:20 AM, Karthick <ibmkarthickma...@gmail.com> >> wrote: >> >>> Hi Team, >>> >>> I'm using fields grouping for a bolt to maintain field-based ordering, >>> but I'm facing data skewness among the bolt's executors. I have 96 >>> executors, and I'm sending data with 500 distinct fields used in the fields >>> grouping. While reviewing the Storm UI, I noticed that a few executors are >>> underutilized while others are overutilized. >>> >>> This seems to be a hashing problem i guess. Can anyone suggest a better >>> hashing technique or approach to resolve this issue? >>> >>> Thanks in advance for your help. >>> >>
