Suppose that you have a bolt whose tasks are not thread safe but you
still want parallelism. It seems that this could be achieved via
multiprocessing by forcing a maximium of 1 executor per worker. With
this constraint, if you chose a parallelism hint of 4 (with default
executors) you would get
Hi Marc,
I believe keeping the total number of executors(i.e. parallelism) across
all the components(bolts,spouts) to be less than or equal to the total
number of workers can be one way to achieve this.
Thanks
Bijoy
On Wed, Feb 5, 2014 at 9:36 PM, Marc Vaillant vaill...@animetrics.comwrote:
You can also write your own scheduler and assign your non-thread-safe bolt
tasks to different worker processes. Then, use the EvenScheduler to
schedule the rest of the tasks. But note that EvenScheduler will not assign
any other tasks to those worker processes (the ones that you assigned).
There
