RE: Storm Topology Hanging

2016-04-10 Thread Nitin Gupta
Hi Alex, Thanks for the guidance! Is there any formula to determine what should be the ideal Max Spout Pending based on the number of tuples per second and the latency time? I read some blogs where the same problem occurred because the max spout pending was too high. The worker log doesn’t s

Re: Message flow in Shuffle grouping

2016-04-10 Thread anshu shukla
Hey Erik Thanks for reply , Actually, I am working on custom scheduler for storm where we can explicitly specify where to place the thread/executor to particular slot mapping on a VM . In order to improve N/W communication cost btw threads .In some papers, I found like if DAG is - [s0-

Re: Message flow in Shuffle grouping

2016-04-10 Thread Erik Weathers
hi Anshu. Can you please clarify what is "not clear" to you in the description? On Sun, Apr 10, 2016 at 2:31 AM, anshu shukla wrote: > Thanks Xiang, > > I have read it already . But I am not clear about this point - > > >1. *Local or shuffle grouping: If the target bolt has one or more >

Re: Message flow in Shuffle grouping

2016-04-10 Thread anshu shukla
Thanks Xiang, I have read it already . But I am not clear about this point - 1. *Local or shuffle grouping: If the target bolt has one or more tasks in the same worker process, tuples will be shuffled to just those in-process tasks. Otherwise, this acts like a normal shuffle grouping*