Hi Vikas,
(I'm newbie) IMO you should inject "storm.local.hostname:" in conf/storm.yaml in the script that starts supervisor. Make sure you inject the FULL name given by "hostname -f" since on ec2 the shorter name is not guaranteed to work (and Java by default gets the short name). We had the same issue with the zookeeper cluster. Since you are probably running a zookeeper cluster, peek at that devops code, a similar solution is probably there. Regards, Itai ________________________________ From: Vikas Agarwal <[email protected]> Sent: Thursday, October 16, 2014 8:04 AM To: [email protected] Subject: Re: Supervisors fail to start up Are you using EC2 private IPs anywhere in your configuration? EC2 internal IP changes after restart. On Thu, Oct 16, 2014 at 1:29 AM, Ryan Ebanks <[email protected]<mailto:[email protected]>> wrote: I have been running a storm-0.8.2 cluster for over a year now. Last night AWS restarted the supervisor machines. I have tried to restart the supervisor processes manually, but upon start up I receive this error message in the logs. 014-10-15 19:48:04 supervisor [ERROR] Error on initialization of server mk-supervisor java.net.UnknownHostException: domU-12-31-39-09-D2-92: domU-12-31-39-09-D2-92 at java.net.InetAddress.getLocalHost(InetAddress.java:1454) at backtype.storm.util$local_hostname.invoke(util.clj:153) at backtype.storm.daemon.supervisor$supervisor_data.invoke(supervisor.clj:181) at backtype.storm.daemon.supervisor$fn__4729$exec_fn__1200__auto____4730.invoke(supervisor.clj:331) at clojure.lang.AFn.applyToHelper(AFn.java:167) at clojure.lang.AFn.applyTo(AFn.java:151) at clojure.core$apply.invoke(core.clj:601) at backtype.storm.daemon.supervisor$fn__4729$mk_supervisor__4754.doInvoke(supervisor.clj:327) at clojure.lang.RestFn.invoke(RestFn.java:436) at backtype.storm.daemon.supervisor$_launch.invoke(supervisor.clj:477) at backtype.storm.daemon.supervisor$_main.invoke(supervisor.clj:506) at clojure.lang.AFn.applyToHelper(AFn.java:159) at clojure.lang.AFn.applyTo(AFn.java:151) at backtype.storm.daemon.supervisor.main(Unknown Source) I am not a clojure expert, but it looks like on line 215 of backtype.storm.daemon.supervisor.clj, that it is possible to set the localhost name in a config file. 215 :my-hostname (if (contains? conf STORM-LOCAL-HOSTNAME) 216 (conf STORM-LOCAL-HOSTNAME) 217 (local-hostname)) Is this possible? What file do I need to set this setting in? What is the correct key for this setting? Or am I way off base and need to do something else to get my workers to restart? Thanks, Ryan Ebanks -- Regards, Vikas Agarwal 91 - 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
