Hello Matthias and thank you for your reply. See my answers below: - I have a 4 supervisor nodes in my AWS cluster of m4.xlarge instances (4 cores per node). On top of that I have 3 more nodes for zookeeper and nimbus. - 2 worker nodes per supervisor node - The task number for each bolt ranges from 1 to 4 and I use 1:1 task to executor assignment. - The number of executors in total for the topology ranges from 14 to 41
Thanks, Nick 2015-09-02 15:42 GMT-04:00 Matthias J. Sax <[email protected]>: > Without any exception/error message it is hard to tell. > > What is your cluster setup > - Hardware, ie, number of cores per node? > - How many node/supervisor are available? > - Configured number of workers for the topology? > - What is the number of task for each spout/bolt? > - What is the number of executors for each spout/bolt? > > -Matthias > > On 09/02/2015 08:02 PM, Nick R. Katsipoulakis wrote: > > Hello all, > > > > I am working on a project in which I submit a topology to my Storm > > cluster, but for some reason, some of my tasks do not start executing. > > > > I can see that the above is happening because every bolt I have needs to > > connect to an external server and do a registration to a service. > > However, some of the bolts do not seem to connect. > > > > I have to say that the number of tasks I have is larger than the number > > of workers of my cluster. Also, I check my worker log files, and I see > > that the workers that do not register, are also not writing some > > initialization messages I have them print in the beginning. > > > > Any idea why this is happening? Can it be because my resources are not > > enough to start off all of the tasks? > > > > Thank you, > > Nick > > -- Nikolaos Romanos Katsipoulakis, University of Pittsburgh, PhD candidate
