Re: Recovering From Zookeeper Failure

2014-05-16 Thread Ryan Chan
Hi Josh We are having the same issue for long time, and only solution is restart the whole storm cluster. (Actually I have asked the same question on 12 May but got no response.) In the meantime, we are currently evaluating switch to Apache Spark for streaming, you might also have a look. On

Lost connection to zookeeper and lead to supervisor restart

2014-05-12 Thread Ryan Chan
This morning, the supervisors/nimbus connections to zookeeper are having problems (they are inside AWS VPC same subnet, not sure the root issue), from the supervisor log: http://pastebin.com/CVMTrfuQ The supervisord died at line: 2014-05-11 00:11:07 b.s.util [INFO] Halting process: ("Error when

Questions on storm configs

2014-02-06 Thread Ryan Chan
Hello, If I deploying nimbus, ui and supervisor on different machine...Do I need to keep the same config file (storm.yaml) for all these nodes? Becasue I have found that I need to place the config such as "supervisor.childopts" on my nimbus storm.yaml, it is a little bit confusing as it should be

Re: No way to specify config file (storm.yml) location

2014-01-31 Thread Ryan Chan
ml", > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 336c337 > < "-Dlogback.configurationFile=" + STORM_DIR + > "/logback/cluster.xml" > --- > > "-Dlogback.configurationFile=" + LOGBACK_FILE > 421a423,428

No way to specify config file (storm.yml) location

2014-01-25 Thread Ryan Chan
There is an old post also talking about this and no useful reply (1+ year ago): https://groups.google.com/forum/#!msg/storm-user/51wualp1xUk/ywkWaMWiLCcJ It mentioned to use the following option if we want to launch the supervisor, e.g. java -client -Dstorm.options= -Dstorm.conf.file=/app/shared/