Hi Azeez, I agree with what you say, and this should reduce much closer to 0%. But, I see some differences in our profile.
In the profile that we attached, when monitoring is disabled, there are some modules consuming more than 3%. I want to see if there is an issue in our profiling approach. Could you attach a new profile for our reference with the new values you speak of? On Tue, Sep 4, 2012 at 10:47 PM, Afkham Azeez <az...@wso2.com> wrote: > Folks, > We need to get this to under 1% overhead by tomorrow. Throughout today, we > have made major performance improvements. Code segment have been improved > to yield from 20% to less than 1% overhead on the request-response path. > Several segments were tuned from 5-10% down to <1% overhead, so I believe > the same can be done for BAM. > > Azeez > > > On Tue, Sep 4, 2012 at 9:47 PM, Afkham Azeez <az...@wso2.com> wrote: > >> Echo service. 40k requests. >> >> >> On Tue, Sep 4, 2012 at 9:44 PM, Tharindu Mathew <thari...@wso2.com>wrote: >> >>> Sinthuja, >>> >>> Can you attach a profile with only stats enabled? The one you attached >>> has both stats and activity enabled. There also seems to be some more >>> improvements we can make to the data publisher side. We can offload more as >>> Azeez suggests. >>> >>> Azeez, for how many requests did you profile for your result and to what >>> service? >>> >>> >>> >>> On Tue, Sep 4, 2012 at 9:38 PM, Afkham Azeez <az...@wso2.com> wrote: >>> >>>> >>>> >>>> On Tue, Sep 4, 2012 at 4:34 PM, Tharindu Mathew <thari...@wso2.com>wrote: >>>> >>>>> The % seems to be quite acceptable after removing registry lookup. I >>>>> will commit this patch. >>>> >>>> >>>> I don't agree with this. 32% overhead for publishing stats is not >>>> acceptable. This indicates that the thread pool which is used for serving >>>> requests is being used for publishing stats, instead of getting the >>>> response back to the client as soon as possible. If you are doing such >>>> heavy lifting, it needs to be offloaded to some other thread pool, which >>>> will take care of publishing separately. >>>> >>>> I would consider this a major blocker for this release. >>>> >>>> >>>>> >>>>> >>>>> On Tue, Sep 4, 2012 at 3:56 PM, Sinthuja Ragendran >>>>> <sinth...@wso2.com>wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have changed the AS publisher to use the local caching. Since the >>>>>> branch is frozen i have attached the patch in JIRA [1]. >>>>>> >>>>>> I did the jprofiling after using the local caching, both profiles >>>>>> when enabling the publishing and not enabling the publishing are >>>>>> attached. >>>>>> >>>>>> [1] https://wso2.org/jira/browse/BAM-821 >>>>>> >>>>>> Thanks, >>>>>> Sinthuja. >>>>>> >>>>>> >>>>>> On Tue, Sep 4, 2012 at 1:58 PM, Afkham Azeez <az...@wso2.com> wrote: >>>>>> >>>>>>> >>>>>>> >>>>>>> On Tue, Sep 4, 2012 at 1:33 PM, Srinath Perera <srin...@wso2.com>wrote: >>>>>>> >>>>>>>> Best is to check this at the time we create the handler and >>>>>>>> remember it >>>>>>> >>>>>>> >>>>>>> +1 >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Sep 4, 2012 at 1:11 PM, Tharindu Mathew >>>>>>>> <thari...@wso2.com>wrote: >>>>>>>> >>>>>>>>> The local caching should also solve the problem of the initial >>>>>>>>> lookup as well. That is we will only consult the registry to see >>>>>>>>> whether >>>>>>>>> stats or activity is enabled ONCE, after that it has to just pass >>>>>>>>> through. >>>>>>>>> >>>>>>>>> Note that the best approach is to not engage the handlers at all, >>>>>>>>> unless stats or activity monitoring is enabled. That should solve >>>>>>>>> problems >>>>>>>>> when monitoring is not required, and the local caching would solve the >>>>>>>>> problem when monitoring is required. >>>>>>>>> >>>>>>>>> Sinthuja, after the fixes please profile and attach a diagram >>>>>>>>> again. >>>>>>>>> >>>>>>>>> On Tue, Sep 4, 2012 at 12:54 PM, Sinthuja Ragendran < >>>>>>>>> sinth...@wso2.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I profiled the as publishing. Since we are using the registry >>>>>>>>>> caching to save the configuration, it is taking more time to >>>>>>>>>> puublish. I >>>>>>>>>> have attached the screen shot of the profiling. >>>>>>>>>> >>>>>>>>>> As per offline discussion with azeez and tharindu, now we thought >>>>>>>>>> of going with local caching as earlier implementation. >>>>>>>>>> And now I'm working on this to reverting to back to previous >>>>>>>>>> implementation. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Sinthuja. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Sep 4, 2012 at 12:44 PM, Suhothayan Sriskandarajah < >>>>>>>>>> s...@wso2.com> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Sep 4, 2012 at 12:00 PM, Srinath Perera < >>>>>>>>>>> srin...@wso2.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> IMHO, we need a urgent code review on this. Also get Suho. >>>>>>>>>>>> >>>>>>>>>>>> Sure, please let me know when the code review is. >>>>>>>>>>> >>>>>>>>>>> Earlier Appserver BAM publishers are not Async, hence Sinthuja >>>>>>>>>>> wrote a AsyncDataPruclihser to solve this issue. Now the data >>>>>>>>>>> publishing >>>>>>>>>>> part is very fast because all the requests are handled through a >>>>>>>>>>> thread >>>>>>>>>>> pool. >>>>>>>>>>> But as I know the data collection, and the event creation logic >>>>>>>>>>> are still not Async, this might me the reason, we have to look into >>>>>>>>>>> this >>>>>>>>>>> more. >>>>>>>>>>> >>>>>>>>>>> Suho >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> We also saw ESB stat collectors looping. >>>>>>>>>>>> >>>>>>>>>>>> --Srinath >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Sep 4, 2012 at 11:46 AM, Afkham Azeez >>>>>>>>>>>> <az...@wso2.com>wrote: >>>>>>>>>>>> >>>>>>>>>>>>> We profiled AppServer 4.1.0 & compared it with the new >>>>>>>>>>>>> AppServer packs after removing the BAM publisher components. The >>>>>>>>>>>>> old >>>>>>>>>>>>> AppServer 4.1.0 is about 10-15% faster! >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Sep 4, 2012 at 11:13 AM, Afkham Azeez >>>>>>>>>>>>> <az...@wso2.com>wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> After removing the BAM publisher components, the throughput >>>>>>>>>>>>>> increased by 10 times! >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Sep 4, 2012 at 11:04 AM, Afkham Azeez <az...@wso2.com >>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please take a look at the attached JProfiler screenshot. >>>>>>>>>>>>>>> This is the contributor to the 50% loss in throughout between >>>>>>>>>>>>>>> the previous >>>>>>>>>>>>>>> AS release and the current AS nightly build. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 >>>>>>>>>>>>>>> 3320919 >>>>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>>>> twitter: >>>>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>>>> * >>>>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>>>> * >>>>>>>>>>>>>>> * >>>>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 >>>>>>>>>>>>>> 3320919 >>>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>>> twitter: >>>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>>> * >>>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>>> * >>>>>>>>>>>>>> * >>>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Afkham Azeez* >>>>>>>>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>>>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>>>>>>>> * <http://www.apache.org/>** >>>>>>>>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 >>>>>>>>>>>>> 3320919 >>>>>>>>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>>>>>>>> twitter: >>>>>>>>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>>>>>>>> * >>>>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>>>>>>>> * >>>>>>>>>>>>> * >>>>>>>>>>>>> *Lean . Enterprise . Middleware* >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> ============================ >>>>>>>>>>>> Srinath Perera, Ph.D. >>>>>>>>>>>> http://www.cs.indiana.edu/~hperera/ >>>>>>>>>>>> http://srinathsview.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Dev mailing list >>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> *S. Suhothayan >>>>>>>>>>> * >>>>>>>>>>> Software Engineer, >>>>>>>>>>> Data Technologies Team, >>>>>>>>>>> *WSO2, Inc. **http://wso2.com >>>>>>>>>>> <http://wso2.com/>* >>>>>>>>>>> *lean.enterprise.middleware.* >>>>>>>>>>> >>>>>>>>>>> *email: **s...@wso2.com* <s...@wso2.com>* cell: (+94) 779 756 >>>>>>>>>>> 757 >>>>>>>>>>> blog: >>>>>>>>>>> **http://suhothayan.blogspot.com/*<http://suhothayan.blogspot.com/> >>>>>>>>>>> * >>>>>>>>>>> twitter: >>>>>>>>>>> **http://twitter.com/suhothayan*<http://twitter.com/suhothayan> >>>>>>>>>>> * >>>>>>>>>>> linked-in: **http://lk.linkedin.com/in/suhothayan* >>>>>>>>>>> * >>>>>>>>>>> * >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Dev mailing list >>>>>>>>>>> Dev@wso2.org >>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Tharindu >>>>>>>>> >>>>>>>>> blog: http://mackiemathew.com/ >>>>>>>>> M: +94777759908 >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> ============================ >>>>>>>> Srinath Perera, Ph.D. >>>>>>>> Senior Software Architect, WSO2 Inc. >>>>>>>> Visiting Faculty, University of Moratuwa >>>>>>>> Member, Apache Software Foundation >>>>>>>> Research Scientist, Lanka Software Foundation >>>>>>>> Blog: http://srinathsview.blogspot.com/ >>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>> Phone: 0772360902 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Afkham Azeez* >>>>>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>>>>> Member; Apache Software Foundation; http://www.apache.org/ >>>>>>> * <http://www.apache.org/>** >>>>>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>>>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>>>>> twitter: >>>>>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>>>>> * >>>>>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>>>>> * >>>>>>> * >>>>>>> *Lean . Enterprise . Middleware* >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> >>>>> Tharindu >>>>> >>>>> blog: http://mackiemathew.com/ >>>>> M: +94777759908 >>>>> >>>>> >>>> >>>> >>>> -- >>>> *Afkham Azeez* >>>> Director of Architecture; WSO2, Inc.; http://wso2.com >>>> Member; Apache Software Foundation; http://www.apache.org/ >>>> * <http://www.apache.org/>** >>>> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >>>> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >>>> twitter: >>>> **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >>>> * >>>> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >>>> * >>>> * >>>> *Lean . Enterprise . Middleware* >>>> >>>> >>> >>> >>> -- >>> Regards, >>> >>> Tharindu >>> >>> blog: http://mackiemathew.com/ >>> M: +94777759908 >>> >>> >> >> >> -- >> *Afkham Azeez* >> Director of Architecture; WSO2, Inc.; http://wso2.com >> Member; Apache Software Foundation; http://www.apache.org/ >> * <http://www.apache.org/>** >> email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 >> blog: **http://blog.afkham.org* <http://blog.afkham.org>* >> twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> >> * >> linked-in: **http://lk.linkedin.com/in/afkhamazeez* >> * >> * >> *Lean . Enterprise . Middleware* >> >> > > > -- > *Afkham Azeez* > Director of Architecture; WSO2, Inc.; http://wso2.com > Member; Apache Software Foundation; http://www.apache.org/ > * <http://www.apache.org/>** > email: **az...@wso2.com* <az...@wso2.com>* cell: +94 77 3320919 > blog: **http://blog.afkham.org* <http://blog.afkham.org>* > twitter: **http://twitter.com/afkham_azeez*<http://twitter.com/afkham_azeez> > * > linked-in: **http://lk.linkedin.com/in/afkhamazeez* > * > * > *Lean . Enterprise . Middleware* > > -- Regards, Tharindu blog: http://mackiemathew.com/ M: +94777759908
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
