Java by default also tries to keep running if something unexpected happens. If an exception is not caught and goes all the way up the stack, that thread will just print out a stack trace to stderr and then quietly stop running. Do you know how many zombie daemons I have had to debug/fix because something slightly unexpected happened on a critical thread. I personally added in a generic uncaught exception handler to all of the Hadoop daemons years ago because it was happening all of the time in the early days of YARN. Storm is almost 100% immune to those situations, and has been since the beginning. That is not to say that storm is not stable. On most of our clusters the only time nimbus or a supervisor goes down is when we upgrade it. We have had daemons running for months. And if they do go down they typically recover in a few seconds, and my team gets an alert so we can debug what happened. Running storm (and I would argue any critical daemon) under supervision for production environments is just a best practice. For debugging/testing something then go ahead and use nohup.
- Bobby On Friday, August 4, 2017, 9:43:10 AM CDT, J.R. Pauley <[email protected]> wrote: thanks for all the responses. screen and nohup seem easy to adopt. I guess for something intended to run on large clusters I'm surprised at the default behavior of storm. It seems quite the opposite of what I would've guessed. I also have a Heron instance running in similar fashion and it's default behavior is to keep running which seems more what I expected. On Fri, Aug 4, 2017 at 9:28 AM, M. Aaron Bossert <[email protected]> wrote: Try adding nohup at the beginning of each of those commands. Nohup prevents a process from terminating when it's parent shell is closed. nohup /opt/storm/apache*/bin/ storm nimbus & Sent from my iPhone On Aug 4, 2017, at 09:08, Ethan Li <[email protected]> wrote: Hi, Storm docs recommend to use daemontools or monit which provides error recovery and etc.. I use linux "nohup" or "screen" command for simplicity. Ethan On Friday, August 4, 2017 7:29 AM, J.R. Pauley <[email protected]> wrote: this seems silly but I have not figured out how to keep nimbus, supervisor running after console session ends. zookeeper survives but supervisor and nimbus shut down when the console session ends. I have storm 1.0.2 installed under /opt/storm and starting up as:/opt/storm/apache*/bin/storm nimbus&/opt/storm/apache*/bin/storm supervisor& /opt/storm/apache*/bin/storm ui& /opt/storm/apache*/bin/storm drpc& I know I can add a while loop to keep the console active but there has to be a better way
