Re: MemberFault event is lost forever when MB is down

2014-09-18 Thread Michiel Blokzijl (mblokzij)
Hi, I’m guessing the fix for [1] will be in 4.1.0, right? I’m glad you managed to resolve both issues with 1 fix! Thanks and best regards, Michiel [1] https://issues.apache.org/jira/browse/STRATOS-795 On 12 Sep 2014, at 15:57, Lahiru Sandaruwan wrote: > Ok cool, Let's resolve the Jira. >

Re: MemberFault event is lost forever when MB is down

2014-09-12 Thread Lahiru Sandaruwan
Ok cool, Let's resolve the Jira. On Fri, Sep 12, 2014 at 5:51 PM, Akila Ravihansa Perera wrote: > Hi Lahiru, > > Yes, this is resolved now. Stratos will now check health stats against > the member list published by CC to topology topic (CompleteTopology > event). This will allow Stratos to recov

Re: MemberFault event is lost forever when MB is down

2014-09-12 Thread Akila Ravihansa Perera
Hi Lahiru, Yes, this is resolved now. Stratos will now check health stats against the member list published by CC to topology topic (CompleteTopology event). This will allow Stratos to recover from MB failures and also server unavailable situations. Thanks. On Fri, Sep 12, 2014 at 5:03 PM, Lahir

Re: MemberFault event is lost forever when MB is down

2014-09-12 Thread Lahiru Sandaruwan
Hi Akila, Would [1] also be solved with the solution we talked here? Thanks. [1] https://issues.apache.org/jira/browse/STRATOS-795 On Thu, Aug 28, 2014 at 12:48 PM, Akila Ravihansa Perera wrote: > Hi, > > Since we're using WSO2 CEP for monitoring faulty members, it would > make sense to enhanc

Re: MemberFault event is lost forever when MB is down

2014-08-28 Thread Akila Ravihansa Perera
Hi, Since we're using WSO2 CEP for monitoring faulty members, it would make sense to enhance the Faulty Member window processor [1] to recover from a core component failure. I have made some improvements to this window processor and committed in [2]. CEP will now have an additional dependency for

Re: MemberFault event is lost forever when MB is down

2014-07-30 Thread Udara Liyanage
Hi Imesh, Yes any message will not be communicated when message broker is not available. On Wed, Jul 30, 2014 at 7:24 PM, Imesh Gunaratne wrote: > As I understood its not just the Member Fault event that is affected in > this scenario, any event that CEP publishes to message broker will > enco

Re: MemberFault event is lost forever when MB is down

2014-07-30 Thread Imesh Gunaratne
As I understood its not just the Member Fault event that is affected in this scenario, any event that CEP publishes to message broker will encounter the same problem. On Wed, Jul 30, 2014 at 5:49 AM, Michiel Blokzijl (mblokzij) < mblok...@cisco.com> wrote: > +1. > > If Stratos, or any component

Re: MemberFault event is lost forever when MB is down

2014-07-30 Thread Michiel Blokzijl (mblokzij)
+1. If Stratos, or any component it relies on, fails, and eventually returns to service, Stratos should "orchestrate" the cloud back to the desired state. If any cartridges went missing and after some time T (post failure) Stratos hasn’t re-discovered them, they should be respawned. Best regar

Re: MemberFault event is lost forever when MB is down

2014-07-29 Thread Isuru Haththotuwa
On Wed, Jul 30, 2014 at 9:45 AM, Akila Ravihansa Perera wrote: > Hi Devs, > > Current Stratos architecture relies heavily on high availability of > the message broker. We faced a situation when MB is down, some of the > messages published will get lost forever and the system state will > never be

Re: MemberFault event is lost forever when MB is down

2014-07-29 Thread Manula Chathurika Thantriwatte
Hi, +1 for having monitoring system. Otherwise it may cause serious problems in production environments. On Wed, Jul 30, 2014 at 9:55 AM, Udara Liyanage wrote: > Hi Akila, > > +1 > The core reason for this is Stratos function is heavily depending on the > message broker. Losing messages, or un

Re: MemberFault event is lost forever when MB is down

2014-07-29 Thread Udara Liyanage
Hi Akila, +1 The core reason for this is Stratos function is heavily depending on the message broker. Losing messages, or unavailability of the MB causes system go into a problematic state. $subject is one of example scenario. We should have a health monitoring system which does not depends on the

MemberFault event is lost forever when MB is down

2014-07-29 Thread Akila Ravihansa Perera
Hi Devs, Current Stratos architecture relies heavily on high availability of the message broker. We faced a situation when MB is down, some of the messages published will get lost forever and the system state will never be recovered. One such example is, when a cartridge instance goes down the CE