Hi Srinath, We are currently running performance tests to identify any performance impact with the data publishing from ESB side. We will run a long running test also and investigate on this behavior and update this thread.
Cheers, Chanaka On Tue, Apr 26, 2016 at 1:23 PM, Srinath Perera <srin...@wso2.com> wrote: > >> >> * This is with 3 ESB nodes publishing data to DAS. So the above graph >> shows the overall TPS for all 3 ESB nodes. The peak to start with is the >> initial TPS we set. Then in decreases gradually and converges to some low >> amount. >> >> @ESB Team: Would like to know is this an expected behavior? >> > This is not exepected bahviour. Chanaka, shall we do a long running test? > >> >> >> Thanks, >> Supun >> >> >> On Wed, Apr 20, 2016 at 12:51 PM, Sriskandarajah Suhothayan < >> s...@wso2.com> wrote: >> >>> +1 for the approach. Lets test it and see. >>> >>> Regards >>> Suho >>> >>> On Wed, Apr 20, 2016 at 12:30 PM, Anjana Fernando <anj...@wso2.com> >>> wrote: >>> >>>> Hi, >>>> >>>> Good progress Supun! .. do keep pushing the parameters to find the >>>> limits we can go to. >>>> >>>> @Suho, the idea was to all together eliminate the batch script and just >>>> store/index the data for later lookup, and do the computation purely in >>>> Siddhi. I don't think we will get a big scaling problem, since the data >>>> needs to be stored in-memory when we go to upper layers of summarization is >>>> smaller, and stops at yearly granularity. So it would be at that time, we >>>> having data in-memory for last years worth of data, in a way of last 12 >>>> records of summary data for 12 months for a specific artifact, last day's >>>> worth, that is 30 entries etc.. so growing of data slows immensely, and >>>> also it has a upper limit, which I guess should be comfortability within >>>> usual memory capacity. >>>> >>>> So if we can get a proper checkpoint and replay mechanism figured out >>>> for data processed, we can do all the things in CEP, then we just don't >>>> have the complexity of maintaining two mechanism of doing the processing. >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> On Wed, Apr 20, 2016 at 12:11 PM, Sriskandarajah Suhothayan < >>>> s...@wso2.com> wrote: >>>> >>>>> I think it will make more sense to run seconds and minutes from >>>>> siddhi, and run the spark every hour, when there are lots of date on the >>>>> system this will be much more scalable. >>>>> >>>>> WDYT? >>>>> >>>>> Regards >>>>> Suho >>>>> >>>>> On Wed, Apr 20, 2016 at 11:50 AM, Supun Sethunga <sup...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> This is a follow-up mail of [1], to give an update on the status with >>>>>> the performance issue [2] . So as mentioned in the previous mail, with >>>>>> Spark-script doing the summary stat generation as a batch process, >>>>>> creates >>>>>> a bottleneck at a higher TPS. More precisely, with our findings, it >>>>>> cannot >>>>>> handle a throughput of more than 30 TPS as a batch process. (i.e: events >>>>>> published to DAS within 10 mins with a TPS of 30, take more than 10 mins >>>>>> to >>>>>> process. Means, if we schedule a script every 10 mins, the events to be >>>>>> processed grows over time). >>>>>> >>>>>> To overcome this, thought of doing the summarizing up to a certain >>>>>> extent (upto second-wise summary) using siddhi, and to generate remaining >>>>>> stats (per-minute/hour/day/month), using spark. With this enhancement, >>>>>> ran >>>>>> some load tests locally to evaluate this approach, and the results are as >>>>>> follows. >>>>>> >>>>>> Backend DB : MySQL >>>>>> ESB analytics nodes: 1 >>>>>> >>>>>> With InnoDB >>>>>> >>>>>> - With *80 TPS*: (script scheduled every 1 min) : Avg time taken >>>>>> for completion of the script = ~ *20 sec*. >>>>>> - With* 500 TPS* (script scheduled every 2 min) : Avg time taken >>>>>> for completion of the script = ~ *45 sec*. >>>>>> >>>>>> >>>>>> With MyISAM >>>>>> >>>>>> - With *80 TPS* (script scheduled every 1 min) : Avg time taken >>>>>> for completion of the script = ~ *24 sec*. >>>>>> - With *80 TPS *(script scheduled every 2 min) : Avg time taken >>>>>> for completion of the script = ~ *20 sec*. >>>>>> - With *500 TPS* (script scheduled every 2 min) : Avg time taken >>>>>> for completion of the script = ~ *35 sec*. >>>>>> >>>>>> As a further improvement, we would be trying out to do summarizing >>>>>> upto minute/hour level (eventually do all the summarizing using siddhi). >>>>>> >>>>>> [1] [Dev] ESB Analytics - Verifying the common production use cases >>>>>> [2] https://wso2.org/jira/browse/ANLYESB-15 >>>>>> >>>>>> Thanks, >>>>>> Supun >>>>>> >>>>>> -- >>>>>> *Supun Sethunga* >>>>>> Software Engineer >>>>>> WSO2, Inc. >>>>>> http://wso2.com/ >>>>>> lean | enterprise | middleware >>>>>> Mobile : +94 716546324 >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> Architecture@wso2.org >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *S. Suhothayan* >>>>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>>>> *WSO2 Inc. *http://wso2.com >>>>> * <http://wso2.com/>* >>>>> lean . enterprise . middleware >>>>> >>>>> >>>>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>>>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>>>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>>>> http://lk.linkedin.com/in/suhothayan >>>>> <http://lk.linkedin.com/in/suhothayan>* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Technical Lead >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>> >>> >>> >>> -- >>> >>> *S. Suhothayan* >>> Technical Lead & Team Lead of WSO2 Complex Event Processor >>> *WSO2 Inc. *http://wso2.com >>> * <http://wso2.com/>* >>> lean . enterprise . middleware >>> >>> >>> *cell: (+94) 779 756 757 <%28%2B94%29%20779%20756%20757> | blog: >>> http://suhothayan.blogspot.com/ <http://suhothayan.blogspot.com/>twitter: >>> http://twitter.com/suhothayan <http://twitter.com/suhothayan> | linked-in: >>> http://lk.linkedin.com/in/suhothayan <http://lk.linkedin.com/in/suhothayan>* >>> >>> _______________________________________________ >>> Architecture mailing list >>> Architecture@wso2.org >>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>> >>> >> >> >> -- >> *Supun Sethunga* >> Software Engineer >> WSO2, Inc. >> http://wso2.com/ >> lean | enterprise | middleware >> Mobile : +94 716546324 >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > > -- > ============================ > Srinath Perera, Ph.D. > http://people.apache.org/~hemapani/ > http://srinathsview.blogspot.com/ > -- Thank you and Best Regards, Chanaka Fernando Senior Technical Lead WSO2, Inc.; http://wso2.com lean.enterprise.middleware mobile: +94 773337238 Blog : http://soatutorials.blogspot.com LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 Twitter:https://twitter.com/chanakaudaya
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
