Could you open a Jira ( blocker)? We need to figure out a solution for the project as well.
--Srinath On Mon, Apr 18, 2016 at 12:17 PM, Supun Sethunga <[email protected]> wrote: > Hi Kasun, > > We ran several rounds of load tests for ESB analytics (covering few > scenarios with several proxy services, APIs, Sequences, etc). In the first > round, we ran the tests with a low TPS (around 10-15) to test and validate > the use cases, such as the counts, delays, drilling down to > mediator/sequence/message level, etc. These scenarios worked as expected. > > On the second round, we ran the same tests with a higher TPS (gradually > increasing). In the process, we identified a bottleneck at the DB level > under a high TPS (TPS of over 50, which is not that high), resulting the > stat generation to be slowed down with time. This occurs as DAS is both > reading and writing from/to the same table (row events) at the same time, > at a rapid rate, causing the IO operations to be slowed down (due to table > locks and etc.), and hence the summarizing process takes longer time, in > each iteration. (most of the consumed time for summarizing, around 80% is > taken for reading the raw data and store them as event-wise data in DB) > Therefore, we are currently doing an enhancement to improve performance, > by summarizing the data (summarize up to per-second info) on the fly with > siddhi and store it, rather than spark doing it as a batch operation. This > should eliminate roughly around 80% (above mentioned) of time taken to > read/write row data from DB. > > Will give an update after re-running the load test with this enhancement. > > Regards, > Supun > > On Mon, Apr 18, 2016 at 11:40 AM, Kasun Indrasiri <[email protected]> wrote: > >> Any updates on this? >> >> We must test the ESB analytics/ ESB 5 alpha release with the common >> production use cases + high load. >> >> On Mon, Mar 28, 2016 at 2:38 PM, Buddhima Wijeweera <[email protected]> >> wrote: >> >>> Hi All, >>> >>> We are currently in the process of testing statistics & tracing for >>> various scenarios. >>> Initially we planned to cover on the areas we have identified for >>> functionality testing. >>> Then we planned to move on to common scenarios, and performance testing >>> as well. >>> >>> Thank you, >>> >>> On Mon, Mar 28, 2016 at 2:34 PM, Srinath Perera <[email protected]> >>> wrote: >>> >>>> Please run ESB benchmark, get the events comes out of it to DAS, and >>>> use that for testing. >>>> >>>> --Srinath >>>> >>>> On Mon, Mar 28, 2016 at 10:02 AM, Kasun Indrasiri <[email protected]> >>>> wrote: >>>> >>>>> Hi Buddhima/VirajS, >>>>> >>>>> Once the end to end functionalities are implemented on the ESB >>>>> analytics side, can we start testing the current implementation for >>>>> production use cases of ESB. Basically we need to cover all the common >>>>> integration scenarios and stress test the implemented with high load too. >>>>> >>>>> Thanks, >>>>> Kasun >>>>> >>>>> -- >>>>> Kasun Indrasiri >>>>> Software Architect >>>>> WSO2, Inc.; http://wso2.com >>>>> lean.enterprise.middleware >>>>> >>>>> cell: +94 77 556 5206 >>>>> Blog : http://kasunpanorama.blogspot.com/ >>>>> >>>> >>>> >>>> >>>> -- >>>> ============================ >>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>> Site: http://home.apache.org/~hemapani/ >>>> Photos: http://www.flickr.com/photos/hemapani/ >>>> Phone: 0772360902 >>>> >>> >>> >>> >>> -- >>> Buddhima Wijeweera >>> Software Engineer; WSO2 Inc.; http://wso2.com , >>> >>> Mobile: +94 71 427 9966 >>> Email: [email protected] >>> Blog: https://buddhimawijeweera.wordpress.com >>> GitHub Profile: https://github.com/Buddhima >>> >> >> >> >> -- >> Kasun Indrasiri >> Software Architect >> WSO2, Inc.; http://wso2.com >> lean.enterprise.middleware >> >> cell: +94 77 556 5206 >> Blog : http://kasunpanorama.blogspot.com/ >> > > > > -- > *Supun Sethunga* > Software Engineer > WSO2, Inc. > http://wso2.com/ > lean | enterprise | middleware > Mobile : +94 716546324 > -- ============================ Blog: http://srinathsview.blogspot.com twitter:@srinath_perera Site: http://home.apache.org/~hemapani/ Photos: http://www.flickr.com/photos/hemapani/ Phone: 0772360902
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
