It surely did! Thanks for such a precint answer! Thanks, Baek
> On Jun 8, 2015, at 12:43 AM, Vineet Mishra <clearmido...@gmail.com> wrote: > > Any Storm Streaming job runs in its own space and doesn't interact with other > topology. Your tuple distribution will be across the topology within the > number of workers on the number of bolts defined, so for instance if you have > shuffle grouping enabled and specific data of your interest > > 0 1 - Kafka Partition > s1 s2 - Subscribed Spouts > b1 b2 b3 b4 - bolts available > > Then all the data(tuple) which is passing through s1 and s2(which indeed are > subscribed to Kafka Partition 0 and 1) are going to emit to bolts b[1-4] > based on the hash of the tuple key so it will be something like, for the data > > tuple(somefancydata1) - b1 > tuple(somefancydata43) - b3 > tuple(somefancydata855) - b1 > > and so on. . .the data(tuple) will be distinct across the bolts! > > Let me know if that solves your concern! > > Cheers! > >> On Mon, Jun 8, 2015 at 11:05 AM, Seungtack Baek >> <seungtackb...@precocityllc.com> wrote: >> @Vineet, >> >> Thanks a lot for "another" timely response! >> >> Actually I have read that section but it wasn't still clear (to me, and I >> guess to me only) whether field grouping was concerning the whole cluster >> (or topology) or for the same worker only.. Maybe I am not too familiar with >> the "zoo". >> >> >> Thanks, >> Baek >> >> >> Seungtack Baek | Precocity, LLC >> Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 >> seungtackb...@precocityllc.com | www.precocityllc.com >> >> This is the end of this message. >> -- >> >>> On Mon, Jun 8, 2015 at 12:31 AM, Vineet Mishra <clearmido...@gmail.com> >>> wrote: >>> Hi Seung, >>> >>> You can better refer to the section Stream Groupings in the following link >>> attached below >>> >>> https://storm.apache.org/documentation/Concepts.html >>> >>> It will get you better understanding of the tuple distribution in Storm, >>> for clear understanding here is the pictorial representation of the same, >>> >>> https://blog.safaribooksonline.com/wp-content/uploads/2013/06/Grouping.png >>> >>> Cheers! >>> >>>> On Mon, Jun 8, 2015 at 10:47 AM, Dima Dragan <dima.dra...@belleron.net> >>>> wrote: >>>> Hi, Seungtack! >>>> >>>> Distribution of messages will be depends only from grouping (in case of >>>> "shuffe grouping", Tuples are randomly distributed across the all bolt's >>>> tasks in a way such that each bolt is guaranteed to get an equal number of >>>> tuples. >>>> >>>> Best regards, >>>> Dmytro Dragan >>>> >>>>> On Jun 8, 2015 07:12, "Seungtack Baek" <seungtackb...@precocityllc.com> >>>>> wrote: >>>>> Hi, >>>>> >>>>> I have read from the documentation that if you have more spout tasks than >>>>> kafka partition, the excessive tasks will remain idle for entire >>>>> lifecycle of the topology. >>>>> >>>>> Now, Let's consider 4 spout tasks, 32 bolt tasks (of one class) in 4 >>>>> workers (in 4 nodes) and 2 partitions in kafka. Then 2 tasks will be >>>>> assigned to each partitions in kafka and the other 2 will remain idle. >>>>> However, does that mean that only the bolts within the same worker will >>>>> get the messages (assuming shuffle grouping)? Or, do the messages get >>>>> emitted to whatever bolt taks available, regardless of which worker? >>>>> >>>>> Thanks, >>>>> Baek >>>>> >>>>> >>>>> Seungtack Baek | Precocity, LLC >>>>> Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 >>>>> seungtackb...@precocityllc.com | www.precocityllc.com >>>>> >>>>> This is the end of this message. >>>>> -- >>>>> >>>>>> On Sun, Jun 7, 2015 at 10:12 PM, Seungtack Baek >>>>>> <seungtackb...@precocityllc.com> wrote: >>>>>> Hi, >>>>>> >>>>>> I have read from the documentation that if you have more spout tasks >>>>>> than kafka partition, the excessive tasks will remain idle for entire >>>>>> lifecycle of the topology. >>>>>> >>>>>> Now, Let's consider 4 spout tasks, 32 bolt tasks (of one class) in 4 >>>>>> workers (in 4 nodes) and 2 partitions in kafka. Then 2 tasks will be >>>>>> assigned to each partitions in kafka and the other 2 will remain idle. >>>>>> However, does that mean that only the bolts within the same worker will >>>>>> get the messages (assuming shuffle grouping)? Or, do the messages get >>>>>> emitted to whatever bolt taks available, regardless of which worker? >>>>>> >>>>>> Thanks, >>>>>> Baek >>>>>> >>>>>> >>>>>> Seungtack Baek | Precocity, LLC >>>>>> Tel/Direct: (972) 378-1030 | Mobile: (214) 477-5715 >>>>>> seungtackb...@precocityllc.com | www.precocityllc.com >>>>>> >>>>>> This is the end of this message. >>>>>> -- >
