Preethini, It is a bit more complex than `nimbus.supervisor.timeout.secs`. Both nimbus and the supervisor are monitoring workers. If the worker crashes the supervisor should relaunch it. If for some reason that does not happen, nimbus will time out the worker and try to reschedule it.
It could be because the supervisor is down. It could be because that supervisor node is badly configured and is not able to launch the worker properly. It could be all kinds of things. Depending on the version of storm that you have scheduling will happen when a topology is submitted, during a rebalance, or from this timer. But in newer versions of storm it relies on the timer to schedule for submission. So if you disable the timer your topology will not be scheduled until after someone manually re-balances something. The "config" to disable the background thread is NIMBUS-DO-NOT-REASSIGN. I don't know if it has changed with older versions of storm, I would have to dig into the code to know for sure. But there are no tests using it any more so we are likely to remove it as some point in the future. If you want to decrease the frequency of the thread running you can control that by setting nimbus.monitor.freq.secs. Why is rescheduling a problem? If the worker is still heart beating into zookeeper then the scheduler should ignore that worker. - Bobby On Wednesday, July 5, 2017, 1:01:45 PM CDT, preethini v <[email protected]> wrote: Hi, Thanks for the reply. Are we talking about "nimbus.supervisor.timeout.secs" which can be configured in storm.yaml? Bobby, as you mentioned, There are ways to change this, but it is really only for testing. Could you please give further pointers on how this can be achieved? Thanks,Preethini On Wed, Jul 5, 2017 at 5:46 PM, Bobby Evans <[email protected]> wrote: Yes schedule is called regularly. It is called with a 10 second sleep in between by default. There are ways to change this, but it is really only for testing. We want to be sure that if a supervisor crashes that things can be rescheduled in a timely manor. - Bobby On Tuesday, July 4, 2017, 11:59:59 PM CDT, anshu shukla <[email protected]> wrote: Nimbus calls the scheduler after every timeout time. I have an implementation of the custom scheduler that reads the mapping from JSON file and enacts the mapping. In the case of rebalancing with updated mapping, it will enact the new mapping. On Tue, Jul 4, 2017 at 10:25 PM, preethini v <[email protected]> wrote: Hi, I have a question about Storm Default Scheduler. The schedule() function is called when the topology is submitted to the Storm cluster for the first time. I assume that once the schedule() is called, it is called again only during rebalance (ie, if we add more worker nodes) Does Storm call the schedule() again periodically for better reassignments when there is no rebalance or any change to the cluster? Thanks,Preethini -- Thanks & Regards, Anshu Shukla
