Thanks Derek, that makes sense for me! Br, -Tao
-----Original Message----- From: Derek Dagit [mailto:der...@yahoo-inc.com] Sent: Friday, August 22, 2014 3:34 PM To: user@storm.incubator.apache.org Subject: Re: Create multiple supervisors on same node > I also tried another scenario: instead of copying the entire storm home > directory, I only use one storm home, but different storm-local dir and > ports, which both are specified in storm.yaml, I can still create multiple > supervisors. (Of course, every time before I start a new supervisor, I have > to update the storm.yaml for different storm-local dir and ports). You will have two supervisors writing to the same log. I recommend creating two distinct storm home directories unless you have a good reason to have them shared. I think the code assumes it is the only supervisor writing in storm home. -- Derek On 8/22/14, 14:08, Yu, Tao wrote: > Thanks Harsha! > > Just cleaned zookeeper data (stop and re-start zookeeper) and tried again, > now I can create multiple supervisors successfully! > > I also tried another scenario: instead of copying the entire storm home > directory, I only use one storm home, but different storm-local dir and > ports, which both are specified in storm.yaml, I can still create multiple > supervisors. (Of course, every time before I start a new supervisor, I have > to update the storm.yaml for different storm-local dir and ports). > > So my new questions are: > > 1) what's the best approach to create multiple supervisors on same node: > > a) each supervisor has it's own storm home directory; Or > b) all supervisors have common storm home directory. > > in both approaches, supervisors have its own storm-local dir and ports. > > 2) when start supervisor, can we specify storm to use custom configuration > (.yaml)? For example, like: > > $bin/storm supervisor --config conf/myConfig.yaml > > Seems like storm will always use conf/storm.yaml, and I do not see any > document mentions about specifying custom config file. > > Thanks, > -Tao > > -----Original Message----- > From: Harsha [mailto:st...@harsha.io] > Sent: Friday, August 22, 2014 12:57 PM > To: user@storm.incubator.apache.org > Subject: Re: Create multiple supervisors on same node > > Tao, > I tried the above steps I am able to run two supervisors on the > same node. Did you check the logs for supervisor under storm2. If > it didn't created a local_dir/storm dir than your supervisor > daemon might not be running. check for logs if there are any > errors. > -Harsha > > On Fri, Aug 22, 2014, at 09:20 AM, Yu, Tao wrote: >> Thanks Harsha! >> >> I tried your way, and here is what I have (major parts) in my storm.yaml: >> >> storm.local.dir: "/opt/grid/tao/storm/storm-0.8.2/local_data/storm" >> supervisor.slots.ports: >> - 6700 >> - 6701 >> >> 1) I created the 1st supervisor, and I can see specified sub-folder >> "local_data/storm/supervisor" was created under" >> opt/grid/tao/storm/storm-0.8.2". That's OK! >> >> 2) then I copied the entire "storm-0.8.2" folder to a new "storm2" >> ("/opt/grid/tao/storm/storm2") >> >> 3) delete the sub-folder "local_data" under "storm2" >> >> 4) updated the storm.yaml under "storm2" with below change: >> >> storm.local.dir: "/opt/grid/tao/storm/storm2/local_data/storm" >> supervisor.slots.ports: >> - 8700 >> - 8701 >> >> 5) under "storm2", create a new supervisor. >> >> Then the new supervisor still has the 1st supervisor's ID. And under >> "storm2", the sub-folder "local_data/storm" was not created. >> >> Does storm still use the 1st storm home directory >> ("storm/storm-0.8.2") "local_data" folder? >> >> Thanks, >> -Tao >> >> -----Original Message----- >> From: Harsha [mailto:st...@harsha.io] >> Sent: Friday, August 22, 2014 11:28 AM >> To: user@storm.incubator.apache.org >> Subject: Re: Create multiple supervisors on same node >> >> Tao, >> you need to delete the storm-local dir under your copied over storm >> dir ( "storm2"). Otherwise it will still pick up the same >> supervisor-id. >> -Harsha >> >> On Fri, Aug 22, 2014, at 08:16 AM, Yu, Tao wrote: >>> Thanks Derek! >>> >>> I tried your suggestion, copied the entire storm home directory >>> (which, in my case, is "storm-0.8.2") to a new directory "storm2", >>> then in "storm2" directory, I changed the conf/storm.yaml with >>> different ports, and tried to create a new supervisor. Still, got >>> the same supervisor ID as the 1st one (which I created from "storm-0.8.2" >>> directory). >>> >>> Did I do anything incorrectly? >>> >>> -Tao >>> >>> -----Original Message----- >>> From: Derek Dagit [mailto:der...@yahoo-inc.com] >>> Sent: Friday, August 22, 2014 11:01 AM >>> To: user@storm.incubator.apache.org >>> Subject: Re: Create multiple supervisors on same node >>> >>> The two supervisors are sharing the same state, and that is how they >>> get the same randomly-generated ID. >>> >>> If I recall correctly, the default state directory is created in the >>> current working directory of the process, so that is whatever >>> directory you happen to be in when you start the supervisor. >>> >>> I think probably a good thing to do is copy the entire storm home >>> directory, change the storm.yaml in the copy to be configured with >>> different ports as you tried, and make sure to cd into the >>> appropriate directory when you launch the supervisor. >>> >>> -- >>> Derek >>> >>> On 8/22/14, 9:49, Yu, Tao wrote: >>>> Hi all, >>>> >>>> Anyone knows what's the requirement to generate multiple supervisors on >>>> the same node (for same topology)? I can create the 1st supervisor, then >>>> I update the "supervisor.slots.ports" to different ports, and tried to >>>> create the 2nd supervisor on same node, it ends up creating a new >>>> supervisor but with same supervisor ID as the 1st one, so it still only >>>> has one supervisor on that node and storm UI shows 1 supervisor as well. >>>> Any suggestion on how to create the 2nd, 3rd supervisor on the same node? >>>> >>>> Any help is appreciated! >>>> >>>> thanks, >>>> -Tao >>>>
