Re: Can I make Nimbus failsafe
Hi all, can you please guide me the Algo or any inputs how i can make nimbus highly available? I have seen the code of JStorm 9.0 where they make nimbus High available using zookeeper. Any one who is done something in it please guide me. regards Tarkeshwar On Wed, Sep 10, 2014 at 2:16 PM, M.Tarkeshwar Rao tarkeshwa...@gmail.com wrote: Ok we can do it together. How can we plan it? On Tue, Sep 9, 2014 at 6:09 PM, 潘臻轩 zhenxuan...@gmail.com wrote: hi, I can not direct git you source code, it mix our company code, I have do many change for community branch. I will implement it on community branch later, if you interest, do together 2014-09-09 20:05 GMT+08:00 M.Tarkeshwar Rao tarkeshwa...@gmail.com: Hi all, How we can achieve high availability of nimbus. Can you please help us to achieve this? Regards Tarkeshwar
Re: Can I make Nimbus failsafe
I am working on this right now , can’t promise the dates because I have some other tasks as well. The general approach is already described in the JIRA. You basically need 3 things: Leader election between all nimbus hosts. (zookeeper is what I am going with), checks on nimbus background threads and state changing APIS to ensure only leader is changing states. Some way to distribute the local nimbus data directory to all the other nimbus and keep it consistent. Simplest way is to not have a local directory but instead use something like HDFS, if you can take that kind of dependency. Nimbus discovery, could be done either by adding/modifying current rest/thrift APIS or by allowing supervisors or clients to read from zookeeper. I have some code on leader election part with preliminary testing and unit-test here. This is by no means ready for anyone’s use but if you want to take a look to get an idea you can take a look. Thanks Parth On Sep 16, 2014, at 6:04 AM, M.Tarkeshwar Rao tarkeshwa...@gmail.com wrote: something -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Can I make Nimbus failsafe
Hi all, How we can achieve high availability of nimbus. Can you please help us to achieve this? Regards Tarkeshwar
Re: Can I make Nimbus failsafe
I found this https://issues.apache.org/jira/browse/STORM-166 However, I don't understand why its priority is Minor while it would be a major new feature. :) On Tue, Sep 9, 2014 at 5:35 PM, M.Tarkeshwar Rao tarkeshwa...@gmail.com wrote: Hi all, How we can achieve high availability of nimbus. Can you please help us to achieve this? Regards Tarkeshwar -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
Re: Can I make Nimbus failsafe
nimbus is stateless, you can implements nimbus HA feature. I have implements on my branch, I Will contribute to community later 2014-09-09 20:16 GMT+08:00 Vikas Agarwal vi...@infoobjects.com: I found this https://issues.apache.org/jira/browse/STORM-166 However, I don't understand why its priority is Minor while it would be a major new feature. :) On Tue, Sep 9, 2014 at 5:35 PM, M.Tarkeshwar Rao tarkeshwa...@gmail.com wrote: Hi all, How we can achieve high availability of nimbus. Can you please help us to achieve this? Regards Tarkeshwar -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
Re: Can I make Nimbus failsafe
Hi zhenxuanpan Can you help us to achieve this? If you can share the source code then it will be very useful for us. We have to implement it to deliver our product. regards tarkeshwar On Tue, Sep 9, 2014 at 5:57 PM, 潘臻轩 zhenxuan...@gmail.com wrote: nimbus is stateless, you can implements nimbus HA feature. I have implements on my branch, I Will contribute to community later 2014-09-09 20:16 GMT+08:00 Vikas Agarwal vi...@infoobjects.com: I found this https://issues.apache.org/jira/browse/STORM-166 However, I don't understand why its priority is Minor while it would be a major new feature. :) On Tue, Sep 9, 2014 at 5:35 PM, M.Tarkeshwar Rao tarkeshwa...@gmail.com wrote: Hi all, How we can achieve high availability of nimbus. Can you please help us to achieve this? Regards Tarkeshwar -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax
