You can also write your own scheduler and assign your non-thread-safe bolt tasks to different worker processes. Then, use the EvenScheduler to schedule the rest of the tasks. But note that EvenScheduler will not assign any other tasks to those worker processes (the ones that you assigned). There is a tutorial at https://xumingming.sinaapp.com/885/twitter-storm-how-to-develop-a-pluggable-scheduler/
On Wed, Feb 5, 2014 at 8:15 AM, bijoy deb <bijoy.comput...@gmail.com> wrote: > Hi Marc, > > I believe keeping the total number of executors(i.e. parallelism) across > all the components(bolts,spouts) to be less than or equal to the total > number of workers can be one way to achieve this. > > Thanks > Bijoy > > > On Wed, Feb 5, 2014 at 9:36 PM, Marc Vaillant <vaill...@animetrics.com>wrote: > >> Suppose that you have a bolt whose tasks are not thread safe but you >> still want parallelism. It seems that this could be achieved via >> multiprocessing by forcing a maximium of 1 executor per worker. With >> this constraint, if you chose a parallelism hint of 4 (with default >> executors) you would get 4 tasks in 4 executors each running in a >> separate worker. Can this constraint be configured? >> >> Thanks, >> Marc >> > >
