Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-03-21 Thread bijoy deb
Hi, I am running a topology using Storm (version 0.9.1),on a cluster of 3 nodes (3x4=12 slots). My topology has 1 spout(parallelism=2),bolt A (parallelism=2),bolt B (parallelism=1) and bolt C(parallelism=1).Number of tasks (numTasks) for each component is default(1).Number of workers is set as 5.

Re: Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-03-21 Thread Drew Goya
Take a look at "topology.optimize" and "storm.scheduler" I had the same issue and I found that setting topology.optimize to false and storm.scheduler to "backtype.storm.scheduler.EvenScheduler" gave me the even distribution of components I was looking for. On Fri, Mar 21, 2014 at 2:50 AM, bijoy

Re: Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-03-21 Thread Srinath C
There configs are interesting but undocumented on wiki. Thanks for the info. On Fri, Mar 21, 2014 at 10:37 PM, Drew Goya wrote: > Take a look at "topology.optimize" and "storm.scheduler" > > I had the same issue and I found that setting topology.optimize to false > and storm.scheduler to "back

Re: Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-03-21 Thread bijoy deb
Thanks Drew.I am going to try those options and see if that helps. Thanks Bijoy On Fri, Mar 21, 2014 at 10:37 PM, Drew Goya wrote: > Take a look at "topology.optimize" and "storm.scheduler" > > I had the same issue and I found that setting topology.optimize to false > and storm.scheduler to "b

Re: Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-03-21 Thread Nathan Marz
topology.optimize doesn't do anything at the moment. It was something planned for in the early days but turned out to be unecessary. On Fri, Mar 21, 2014 at 9:00 PM, bijoy deb wrote: > Thanks Drew.I am going to try those options and see if that helps. > > Thanks > Bijoy > > > On Fri, Mar 21, 201

Re: Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-03-21 Thread bijoy deb
Thanks Nathan.So, I believe setting storm.scheduler to EvenScheduler,as suggested by Drew should do the trick? However, I still have one doubt.With reference to my use case, what I was looking for is that each component (spout/bolt instance) should get assigned to different slot,since I have free

RE: Multiple Storm components getting assigned to same worker slot despite of free slots being available

2014-05-16 Thread Simon Cooper
r is doing, is there some specification of what exactly EvenScheduler and DefaultScheduler do somewhere? From: bijoy deb [mailto:bijoy.comput...@gmail.com] Sent: 22 March 2014 05:39 To: user@storm.incubator.apache.org Subject: Re: Multiple Storm components getting assigned to same worker slot de