Oh that's good to know. I assume it works like this: https://en.wikipedia.org/wiki/Hash_function#Hashing_uniformly_distributed_data
On Wed, Aug 10, 2016 at 6:23 PM, Nathan Leung <ncle...@gmail.com> wrote: > It's based on a modulo of a hash of the field. The fields grouping is > stateless. > > On Aug 10, 2016 8:18 AM, "Navin Ipe" <navin....@searchlighthealth.com> > wrote: > >> Hi, >> >> For spouts to be able to continuously send a fields grouped tuple to the >> same bolt, it would have to store a key value map something like this, >> right? >> >> field1023 ---> Bolt1 >> field1343 ---> Bolt3 >> field1629 ---> Bolt5 >> field1726 ---> Bolt1 >> field1481 ---> Bolt3 >> >> So if my topology runs for a very long time and the spout generates many >> unique field values, won't this key value map run out of memory eventually? >> >> OR is there a failsafe or a map limit that Storm has to handle this >> without crashing? >> >> If memory problems could happen, what would be an alternative way to >> solve this problem where many unique fields could get generated over time? >> >> -- >> Regards, >> Navin >> > -- Regards, Navin
