Thanks Sahil, Aaron for the reply Are you encountering a performance problem?
Yes facing slowness on the executors with more load. Is the fields grouping absolutely required for your use case? Yes to maintain the ordering it required for my case. Can you suggest some hashes? Does your case needs packets of same hash to go to same executor or > ordering of events ? > Yes, ordering of events is important. Currently we are using a unique id which datatype is bigint(Long). Can you suggest any changes reg this. On Sat, Aug 17, 2024 at 8:40 AM Sahil Kamboj <sahilkamboj...@gmail.com> wrote: > Hey karthick > > Does your case needs packets of same hash to go to same executor or > ordering of events ? If not then go with local or shuffle grouping which > will distribute load in round robin manner across executors. Otherwise > choose your key wisely to avoid skewness when using fields grouping. > > > > On Sat, 17 Aug 2024 at 5:20 AM, Karthick <ibmkarthickma...@gmail.com> > wrote: > >> Hi Team, >> >> I'm using fields grouping for a bolt to maintain field-based ordering, >> but I'm facing data skewness among the bolt's executors. I have 96 >> executors, and I'm sending data with 500 distinct fields used in the fields >> grouping. While reviewing the Storm UI, I noticed that a few executors are >> underutilized while others are overutilized. >> >> This seems to be a hashing problem i guess. Can anyone suggest a better >> hashing technique or approach to resolve this issue? >> >> Thanks in advance for your help. >> >
