Yes, to do what you want you would need to implement a custom scheduler. More details can be found here: http://storm.apache.org/releases/1.0.2/Storm-Scheduler.html <http://storm.apache.org/releases/1.0.2/Storm-Scheduler.html>
-Taylor > On Nov 7, 2016, at 11:31 AM, Arthur Maciejewicz <art...@signafire.com> wrote: > > Hello All, > > Is there a way to make the "shape" of topology parallelism explicit? For > example, assume: > > * W workers on W nodes > * 2 spouts with 1 executor each (total of 2 executors) > * 1 "mapper" bolt with M executors > * 1 "process" bolt with N executors. > > Is it currently possibly to co-locate the spouts with the "mappers" in 2 > workers, while pinning the N "process" executors to the remaining (W-2) > workers? > > Visually this is what I want to do: > > > 2 Workers W-2 Workers > > +-----------+ > +----------+ | process | > | spout | +-----------+ > | + +--> +-----------+ > | map | | process | > +----------+ +-----------+ > +-----------+ > +----------+ | process | > | spout | +-----------+ > | + +--> ... > | map | +-----------+ > +----------+ | process | > +-----------+ > +-----------+ > | process | > +-----------+ > > Thanks, > > Arthur
