RE: Re: How much is the overhead of time to deploy a system on Storm ?

2015-05-14 Thread Rajesh_Kalluri
Dell - Internal Use - Confidential Nathan, Can you explain in a little more detail what you mean by “When you have more tasks than executors, the spout thread does the same logic, it just does it for more tasks during its main loop.” I thought the spout thread emits tuples based on the max

RE: Re: How much is the overhead of time to deploy a system on Storm ?

2015-05-14 Thread Nathan Leung
Maybe I will make an analogy. Think of spout executors as people wrapping presents. Think of spout tasks as tables where people can wrap presents. If you have 10 tasks and 1 executor, then you have 10 tasks and 1 person. The person will wrap a present at one table, then go to the next, wrap a

Re: Re: How much is the overhead of time to deploy a system on Storm ?

2015-05-13 Thread yang...@bupt.edu.cn
I'm not very surprised. See for example published single machine benchmarks (iirc 1.6 million tuples / s is the official figure from Nathan Marz though that figure is a little old). This is spout to bolt and matches my observations for trivial cases. With some processing logic and only one spout I