Great work Akila! This is definitely a tremendous improvement. On Wed, Nov 11, 2015 at 3:27 PM, Akila Ravihansa Perera <raviha...@wso2.com> wrote:
> Hi, > > I've implemented the following improvements and merged in [1]. JIRAs > created [2, 3, 4] to track these tasks. > > - On-demand initialization of complete topology > - On-demand initialization of complete application topology > - On-demand initialization of complete tenant model > - On-demand initialization of complete application signups > - New MB topic named "initializer" and new events to request the above > - Updated PCA to use request events to initialize topology and tenant > model > > - Updated Stratos integration test to verify initialization of complete > topology, application model, tenant model and application signups within > 20s. I've added logs in the test case to print actual time it took to > initialize those. On average it takes less than 1s now. > > - Updated Python agent live test case to verify topology and tenant > models are requested via request events published to initializer topic and > in-memory models are initialized properly. > > [1] > https://github.com/apache/stratos/commit/60b801144dcb05e3664353c412bc1c3fffd7c55c > [2] https://issues.apache.org/jira/browse/STRATOS-1544 > [3] https://issues.apache.org/jira/browse/STRATOS-1612 > [4] https://issues.apache.org/jira/browse/STRATOS-1611 > > Thanks. > > On Tue, Nov 10, 2015 at 9:03 AM, Isuru Haththotuwa <isu...@apache.org> > wrote: > >> >> >> On Tue, Nov 10, 2015 at 8:42 AM, Akila Ravihansa Perera < >> raviha...@wso2.com> wrote: >> >>> Hi, >>> >>> I've updated Python agent to use this method to initialize the topology. >>> Also extended the same logic to initialize complete application model. >>> >>> @Imesh: Sure, will do a thorough test before merging. >>> >>> @Isuru: I think we cannot get rid of complete topology synchronization >>> task (periodically published event), since there can be situations where >>> complete topology request event could be missed due to unavailability of CC >>> component (maintenance restarts, failures etc.). >>> >> +1. For now we can increase the time gap between two complete topology >> events to stop flooding the network unnecessarily. WDYT? >> >>> >>> Also currently Python agent does not have a mechanism to build the >>> topology on-the-fly using topology events. It relies on complete topology >>> event to update its topology context. Perhaps we can think this through >>> more carefully and remove the periodical publish task in the next major >>> release. >>> >>> Thanks. >>> >>> On Tue, Nov 10, 2015 at 6:07 AM, Isuru Haththotuwa <isu...@apache.org> >>> wrote: >>> >>>> Great work Akila. Just to clarify, are we removing the Complete >>>> Topology event and its usages? >>>> >>>> On Tue, Nov 10, 2015 at 4:16 AM, Imesh Gunaratne <im...@apache.org> >>>> wrote: >>>> >>>>> Great work Akila! >>>>> >>>>> Please do a thorough test and make sure everything is covered before >>>>> pushing this change to the current code base. If not I would like to >>>>> suggest to push this to the next major release. >>>>> >>>>> Thanks >>>>> >>>>> On Sun, Nov 8, 2015 at 10:35 PM, Akila Ravihansa Perera < >>>>> raviha...@wso2.com> wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I took the liberty of implementing topology init optimization since >>>>>> that will speed of lot of things that we're trying to do :) >>>>>> >>>>>> Here is my approach; >>>>>> >>>>>> - Introduce a new topic called "initialization" topic >>>>>> - Introduce a new event called "CompleteTopologyRequestEvent" >>>>>> - Create a message receiver, processor, listener for this event >>>>>> - In CC component, create a message receiver object and listen to >>>>>> this CompleteTopologyRequestEvent >>>>>> - Once an event of this type is received, publish the >>>>>> CompleteTopologyEvent >>>>>> >>>>>> *** Update the TopologyReceiver class to publish this >>>>>> CompleteTopologyRequestEvent so that once you start listening to topology >>>>>> events, the underlying messaging component will initialize the topology >>>>>> in >>>>>> an on-demand fashion. >>>>>> >>>>>> I've created a PR [1] for this and still doing further testing. >>>>>> Please go through it and let me know your thoughts. I'm planning on >>>>>> writing >>>>>> an integration test to verify the functionality. I ran a complete build >>>>>> and >>>>>> all test cases are passing. >>>>>> >>>>>> Also Stratos startup time improved very noticeably since other >>>>>> components will not wait for CC to publish the CompleteTopology but >>>>>> rather >>>>>> it will be requested on-demand via messaging component. Once this is >>>>>> stable >>>>>> I'll move on to updating Python agent to use it. >>>>>> >>>>>> [1] https://github.com/apache/stratos/pull/487 >>>>>> >>>>>> Thanks. >>>>>> >>>>>> On Thu, Sep 10, 2015 at 10:27 AM, Pubudu Gunatilaka <pubu...@wso2.com >>>>>> > wrote: >>>>>> >>>>>>> +1 for this approach. I hope PCA plugins will be able to use this. >>>>>>> >>>>>>> On Thu, Sep 10, 2015 at 10:17 AM, Udara Liyanage <ud...@wso2.com> >>>>>>> wrote: >>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> On Thu, Sep 10, 2015 at 10:16 AM, Reka Thirunavukkarasu < >>>>>>>> r...@wso2.com> wrote: >>>>>>>> >>>>>>>>> +1. Great thought. It would increase the startup time of the >>>>>>>>> components those who listens to CompleteTopology and PCA. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Reka >>>>>>>>> >>>>>>>>> On Thu, Sep 10, 2015 at 1:09 AM, Imesh Gunaratne <im...@apache.org >>>>>>>>> > wrote: >>>>>>>>> >>>>>>>>>> Hi Devs, >>>>>>>>>> >>>>>>>>>> I would like to propose $subject. >>>>>>>>>> >>>>>>>>>> Currently we do this with a one minute interval and its causing >>>>>>>>>> the cartridge agent and the servers to wait for around two minutes >>>>>>>>>> (in the >>>>>>>>>> worst case) to start. In Docker this is a considerable amount of >>>>>>>>>> delay. >>>>>>>>>> >>>>>>>>>> I think we can implement $subject by introducing another topic >>>>>>>>>> (say topology-init) and asking the clients to request Complete >>>>>>>>>> Topology >>>>>>>>>> event via the new topic. This can be done in messaging component >>>>>>>>>> without >>>>>>>>>> affecting the main modules (AS, CC, SM, CEP, JCA). PCA needs to be >>>>>>>>>> updated >>>>>>>>>> accordingly. WDYT? >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Imesh Gunaratne >>>>>>>>>> >>>>>>>>>> Senior Technical Lead, WSO2 >>>>>>>>>> Committer & PMC Member, Apache Stratos >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Reka Thirunavukkarasu >>>>>>>>> Senior Software Engineer, >>>>>>>>> WSO2, Inc.:http://wso2.com, >>>>>>>>> Mobile: +94776442007 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Udara Liyanage >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc.: http://wso2.com >>>>>>>> lean. enterprise. middleware >>>>>>>> >>>>>>>> web: http://udaraliyanage.wordpress.com >>>>>>>> phone: +94 71 443 6897 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> *Pubudu Gunatilaka* >>>>>>> Software Engineer >>>>>>> WSO2, Inc.: http://wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> mobile: +94 77 4078049 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Akila Ravihansa Perera >>>>>> WSO2 Inc.; http://wso2.com/ >>>>>> >>>>>> Blog: http://ravihansa3000.blogspot.com >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Imesh Gunaratne >>>>> >>>>> Senior Technical Lead, WSO2 >>>>> Committer & PMC Member, Apache Stratos >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> >>>>> Isuru H. >>>>> +94 716 358 048* <http://wso2.com/>* >>>>> >>>>> >>>>> * <http://wso2.com/>* >>>>> >>>>> >>>>> >>> >>> >>> -- >>> Akila Ravihansa Perera >>> WSO2 Inc.; http://wso2.com/ >>> >>> Blog: http://ravihansa3000.blogspot.com >>> >>> -- >>> <http://ravihansa3000.blogspot.com> >>> <http://ravihansa3000.blogspot.com> >>> Thanks and Regards, >>> >>> Isuru H. >>> <http://ravihansa3000.blogspot.com> >>> +94 716 358 048 <http://ravihansa3000.blogspot.com>* <http://wso2.com/>* >>> >>> >>> * <http://wso2.com/>* >>> >>> >>> > > > -- > Akila Ravihansa Perera > WSO2 Inc.; http://wso2.com/ > > Blog: http://ravihansa3000.blogspot.com > > -- > Thanks and Regards, > > Isuru H. > +94 716 358 048* <http://wso2.com/>* > > > * <http://wso2.com/>* > > >
