Hi Sinthuja, Per min summary within local node will be fine and reduce the traffic to Analyzer. But can we grantee that, Siddhi apps will not slow down other functionalities(gateway request)? This can be try out in ballerina based gateway since streams is already in there. But if we take c4 based products this will not be easy right? we have to release the products with siddhi features.
Thanks and Regards On Tue, Jun 19, 2018 at 5:04 PM, Sinthuja Rajendran <sinth...@wso2.com> wrote: > Hi, > > With my recent work with metrics and other monitoring systems, I'm > thinking whether our model of sending everything and calculate in the > analytics server side is correct. > > Basically IMHO, the majority of the product analytics use cases are > statistics calculation. For example, in APIM, we are calculating API > statistics, Subscription statistics, etc. And for this, we are sending > events for every request/response/fault to analytics server, and analytics > server is performing the actual statistics calculation on the events. > Therefore in the high traffic scenario, there are a lot of events needs to > be published to analytics server, and we are getting issues like "Event > Queue Full" in the gateway. > > My proposal is, what if we calculate the statistics on the local node > itself (similar to the edge analytics by running Siddhi within the > products), and only send the minute statistics summary to analytics server > to do the global (across all nodes) calculation. With this way, the traffic > to analytics server will be reduced drastically as only the final > calculated value for each group by combination will be reported. Therefore, > the analytics server can focus on global summarization and other global > monitoring aspects. > > Thanks, > Sinthuja. > > -- > *Sinthuja Rajendran* > Senior Technical Lead > WSO2, Inc.:http://wso2.com > > Blog: http://sinthu-rajan.blogspot.com/ > Mobile: +94774273955 > > > -- Rukshan Chathuranga. Software Engineer. WSO2, Inc. +94711822074
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
