the best solution is that we can add the multiple nimbus server in the storm.yaml, those should be for failover, it also will be easy to configure
2014-09-09 22:19 GMT-04:00 潘臻轩 <zhenxuan...@gmail.com>: > yes, I have implement this way.. and it ok in fact.. > I implement a total ha solution for nimbus. > and our team write a total scheduler for storm(such as yarn for support > 700+ cluster) > > 2014-09-10 10:02 GMT+08:00 Ankit Toshniwal <ankitoshni...@gmail.com>: > >> Yes, that's a problem area, and we have been discussing it internally on >> how we can handle it better. We are considering moving to an HDFS based >> solution where Nimbus will upload the jars into hdfs instead of local disk >> (as that is a single point of failure) and supervisors will be downloading >> the jar's from hdfs as well. >> >> The other problem we ran into was nic saturation on Nimbus host since too >> many machines were doing copy of the jar's (180MB in size) to worker >> machines leading to the total increase in time. Thus, with moving to HDFS >> based solution we can do this more effectively and faster plus it scales >> better. >> >> We do not have a working prototype for it, but something we are actively >> pursuing. >> >> Ankit >> >> On Tue, Sep 9, 2014 at 6:43 PM, 潘臻轩 <zhenxuan...@gmail.com> wrote: >> >>> I not agree Nathan, if just nimbus down, it is fail-fast.but if the >>> machine happen error(such as disk error), this may lead >>> topology clear. >>> >>> 2014-09-10 9:39 GMT+08:00 潘臻轩 <zhenxuan...@gmail.com>: >>> >>>> *According to my knowledge, is not the case。you should check it with >>>> script or other way.* >>>> >>>> 2014-09-10 0:49 GMT+08:00 Jiang Jacky <jiang0...@gmail.com>: >>>> >>>>> Hi, I read the articles about the nimbus, it specifies the nimbus >>>>> daemon is fail-fast. But I am not sure if it is like Hadoop, there is >>>>> secondary server for failover, if the nimbus server is totally down, then >>>>> the secondary server can be up. Thanks >>>>> >>>> >>>> >>> >> >
