Re: [Architecture] IS-Analytics performance

, "Iranga Muthuthanthri" <ira...@wso2.com> wrote: > > > On Mon, Jun 27, 2016 at 4:35 PM, Mohanadarshan Vivekanandalingam < > mo...@wso2.com> wrote: > >> >> >> On Mon, Jun 27, 2016 at 4:30 PM, Seshika Fernando <sesh...@wso2.com> >>

Re: [Architecture] IS-Analytics performance

Hey saith, This is great. So when you removed the duplicate windows, there were no OOM issues? Seshi On 24 Jun 2016 14:28, "Sajith Ravindra" wrote: > Hi Malith, > > Thanks for the explanation. >> >> I would expect some variation in the throughput. The aim should be >>

Re: [Architecture] [CEP] Siddhi Extension for calculate percentile values

o deal with >>> more than one percentile value (e.g. 90% and 95%). What is the best way to >>> handle case? Can we have an implementation where we can pass an array >>> (containing percentiles) and get the function to return an array of >>> percentile va

Re: [Architecture] [CEP] Siddhi Extension for calculate percentile values

ill return the 97th percentile value of last 100 events of > temperature variable. > > Likewise, we can use other windows such as timeBatch, cron etc as for the > requirement. Would that approach be OK with this? > > Thanks, > Ashen > > On Thu, Jun 9, 2016 at 5:18 AM, Seshik

Re: [Architecture] [CEP] Siddhi Extension for calculate percentile values

Ashen, Is there no way that we can provide a window for temperature field? Practically, we may want to know the 97th percentile of the last n events. In that case how would I use this extension? It might be a good idea to allow user to provide an interval based on length and/or time (like we do

Re: [Architecture] [Dev] WSO2 IS Analytics v1.0.0-ALPHA Released

Hi guys, If we did any changes to the architecture of Authentication analytics, can we update the documentation in [1]. Damith/Mohan: Can one of you take ownership of this? Thanks, Seshi 1.

Re: [Architecture] [CEP] Extending the Regression Function to support time window

orts both time and length, a name such as regressTimeLength > would be appropriate IMO. Please give your suggestions. > > Regards, > Charini > > On Sun, Jun 5, 2016 at 11:28 AM, Seshika Fernando <sesh...@wso2.com> > wrote: > >> Hi, >> The length ceiling i

Re: [Architecture] [Siddhi] Making Disruptor configurable

Hi Suho, Looks good. If (bufferSize='') is optional, what is the default bufferSize that will be taken if I just add @plan:async ? seshi On Sun, May 15, 2016 at 3:58 PM, Sriskandarajah Suhothayan wrote: > > Hi > > We have made Disruptor as optional for Siddhi[1], currently

Re: [Architecture] Moving Custom Components to a common place

Damith, Just for the record, as discussed yesterday, these will be added to analytics.commons repo where we will keep all components that can be used commonly by "Analytics for X" efforts. @Miyuru: Can you add the other components we discussed to be added to this? N.B. analytics.commons - not to

Re: [Architecture] [Analytics] Improvements in search APIs in AnalyticsDataService

Hi Gimantha, This is a very useful functionality. Thanks for adding this. I have 1 question. Can you tell me the difference of result between the following 2 combinations. "sortType" : "DESC" "reversed" : false and "sortType" : "ASC" "reversed" : true My query is, that if you

Re: [Architecture] [Analytics][APIM] - Implement Geo location graph in Analytics

Hi Pulasthi, Can we use this for our IP->Region conversion? seshi On Mon, Mar 21, 2016 at 11:06 AM, Tharindu Dharmarathna wrote: > Hi Janaka and All, > > Here is the performance test result which we have done . > > No of Different IP's : 2000 > > Time Taken to Retrieve and

Re: [Architecture] Request Pattern Change Detection (Analytics for APIM)

i.e. an application >> access a set of APIs in a particular sequence most of the time, but if >> there're multiple applications, API access patterns should be very >> different practically. >> >> On Tue, Feb 16, 2016 at 2:18 PM, Seshika Fernando <sesh...@wso2.com> >>

Re: [Architecture] Request Pattern Change Detection (Analytics for APIM)

rid of the userID even if [2] is the > case. Under the assumption that there will be a lot more genuine users than > fraudulent users? > > > On Tue, Feb 16, 2016 at 11:19 AM, Seshika Fernando <sesh...@wso2.com> > wrote: > >> Hi Fazlan, >> >> Could you expl

Re: [Architecture] Request Pattern Change Detection (Analytics for APIM)

Hi Fazlan, Could you explain the thinking behind assigning userID also as one of the classifiers for the state? The reason I'm asking is that users can learn from each others patterns as well. i.e. if a bunch of users are using the same (or similar) set of apis, they will all follow similar

Re: [Architecture] Packaging analytics artifacts into a p2 repo

+1 On Wed, Feb 10, 2016 at 4:51 AM, Srinath Perera wrote: > Thanks!! Seshika, we should try this with Fraud artifacts. > > --Srinath > > On Tue, Feb 9, 2016 at 11:14 AM, Chanika Geeganage > wrote: > >> Hi, >> >> I have done a POC to evaluate whether it is

Re: [Architecture] Security Analytics

t; Analytics, lets say if there is a fraud detected due to several API Calls > can we find out out from which API calls and calling which backend service? > > On Tue, Feb 2, 2016 at 11:30 AM, Seshika Fernando <sesh...@wso2.com> > wrote: > >> @Lahiru: +1. I'm hoping to anal

Re: [Architecture] Security Analytics

n Mon, Feb 1, 2016 at 11:30 PM, Lahiru Sandaruwan <lahi...@wso2.com> > wrote: > >> Hi Seshi, >> >> I think we can consider Authorization stats also. Since WSO2 IS has a >> good implementation of XACML spec, we can collect stats on, the requests >> allowed,

Re: [Architecture] Notebook Support Use cases for DAS

Hi Srinath, The 'notebooks' that you talk about: are they similar to a sort of staging DAS configuration where we test out/ try out things, and once we are happy we deploy that configuration to the respective pipelines (like Realtime). Cannot access doc. Can you share a 'view only' link? seshi

Re: [Architecture] [ML] Anomaly Detection Feature for WSO2 ML

Welldone Ashen. The documentation looks good too. Will review later. seshi On Thu, Nov 19, 2015 at 10:35 AM, Ashen Weerathunga wrote: > Hi all, > > This feature was implemented on ML and released with WSO2 Machine Learner > 1.1.0 - Milestone 1 >

Re: [Architecture] Connector: SalesforceBulk

Nashry, Please try this out when its done. seshi On Thu, Sep 3, 2015 at 12:02 PM, Malaka Silva wrote: > Great looks good > > On Thu, Sep 3, 2015 at 11:19 AM, Vinitha Rajenthiran < > vinitha.ra...@gmail.com> wrote: > >> Hi Malaka, >> >> Yes, we have covered all the methods. >>

Re: [Architecture] [CEP] Event Simulator Feature

Hi Ramindu, The existing event simulator feature is extremely useful. IMO what would add to its usability is the following... 1. ability to provide a delay between events 2. ability for user to specify how many events to be sent (currently all events in the file will be sent) 3. ability to pause

Re: [Architecture] [DAS] Changing the name of Message Console

My vote is for data explorer for message console and to keep spark console for spark console. seshi On Mon, Jul 13, 2015 at 10:36 AM, Anjana Fernando anj...@wso2.com wrote: Hi, +1 for Data Explorer for message console. The name Spark Console is fine the way it is now. Cheers, Anjana.

Re: [Architecture] [IGVIZ.JS] : D3.js Based Interactive Generic Visualization Library

Thanks, On Thu, Mar 26, 2015 at 7:34 PM, Seshika Fernando sesh...@wso2.com wrote: Hi Tharindu, Can I use aggregate functions with Tables as well? seshi On Thu, Mar 26, 2015 at 5:44 PM, Seshika Fernando sesh...@wso2.com wrote: Oh this is good news indeed. Will use it. On Thu, Mar 26

Re: [Architecture] [IGVIZ.JS] : D3.js Based Interactive Generic Visualization Library

uses the aggregated functions that are supported by vega.js . min,max,sum,avg are the functions that I have tested and they are working . I tested count() but didn't get the result I expected . I 'll fix it . On Thu, Mar 26, 2015 at 1:38 PM, Seshika Fernando sesh...@wso2.com wrote: What

Re: [Architecture] [IGVIZ.JS] : D3.js Based Interactive Generic Visualization Library

variable you can't s use it on Bar charts( which need a discrete domain for X-axis ). For other types of charts like area ,line and scatter you can use it. [1] http://dunithd.github.io/igviz/samples/discreteTime/index.html On Thu, Mar 26, 2015 at 3:57 PM, Seshika Fernando sesh...@wso2.com

Re: [Architecture] [IGVIZ.JS] : D3.js Based Interactive Generic Visualization Library

Hi Tharindu, Can I use aggregate functions with Tables as well? seshi On Thu, Mar 26, 2015 at 5:44 PM, Seshika Fernando sesh...@wso2.com wrote: Oh this is good news indeed. Will use it. On Thu, Mar 26, 2015 at 5:31 PM, Tharindu Munasinghe tharin...@wso2.com wrote: Time domain is already

Re: [Architecture] [IGVIZ.JS] : D3.js Based Interactive Generic Visualization Library

Noted and Thanks. Will try this out. On Thu, Mar 26, 2015 at 1:34 PM, Tharindu Munasinghe tharin...@wso2.com wrote: Hi Seshika, I have updated the igviz.js so that now aggregated functions can be used with bar charts. if you don't use aggregate functions , bar chart will only show

[Architecture] [CEP] Anomaly Detection using Delta Anomaly Detection

Hi all, Inspired by Ted Dunning's talk on Anomaly Detection in [1], we are hoping to implement this in Siddhi. Basically, this is how Delta Anomaly Detection works. a. Use a predictor based on percentiles which predicts the next event based on historical data. b. Obtain the delta between the

Re: [Architecture] Implementing Frequency calculation algorithm in CEP

When considering the usecases for this, getting the frequency during a particular time period will be very useful. (eg:- trading frequency of a different stocks/products during the last 5 hours) Does your custom transformer support this? Or does it always take all events for the frequency

Re: [Architecture] Implementing Frequency calculation algorithm in CEP

:10 PM, Seshika Fernando sesh...@wso2.com wrote: When considering the usecases for this, getting the frequency during a particular time period will be very useful. (eg:- trading frequency of a different stocks/products during the last 5 hours) Does your custom transformer support this? Or does

Re: [Architecture] Extensions for Siddhi - Math operations

Excellent! Some of these will be very useful for the Fraud Detection, and Forecasting algorithms. On Wed, Oct 15, 2014 at 5:34 PM, Lahiru Cooray lahi...@wso2.com wrote: Hi, We have implemented below Math operations (all the Math operations available in Hive) as extensions to Siddhi engine.

[Architecture] Fraud Detection Toolboxes

Hi all, I've started to work on $subject. Fraud Detection is used in many spaces. Some of the most popular ones are credit card fraud, anti money laundering, organizational/operational fraud etc; I'm planning to create toolboxes (for some of these spaces) in CEP, which will work with a defined

Re: [Architecture] Connector:GooglePrediction

Hi Rasika, Do you know what is used to train the models? (regression analysis/ neural networks or something else?) Seshika On Tue, Jun 3, 2014 at 10:06 AM, Rasika Hettige rasi...@gmail.com wrote: Hi Malaka, The final list of methods that will implement under GooglePrediction connector is

[Architecture] Siddhi Time Series Extension - Upper limit and Batch processing

Hi all, There was a issue with the time-series extension where all events keep getting added to the extension and all subsequent regressions consider ALL previous events for the regression calculation. This obviously leads to 'out of memory' issues and the solution was to implement an upper limit

Re: [Architecture] Siddhi Time Seriers Extension - Performance

Hi all, Currently we are performing regression incrementally and we do not have an upperbound (or window size) which limits the number of events that are processed in the regression function. Therefore, we run into memory issues when running performance tests. In order to do a proper performance

Re: [Architecture] Siddhi Time Seriers Extension - Performance

On Mon, Apr 14, 2014 at 10:55 AM, Seshika Fernando sesh...@wso2.comwrote: Hi, This is not over the network. Events are just a couple of data streams joined together (eg:- StockPriceStream, TwitterWordCountStream, MarketIndexStream etc;). This test was performed from our laptops. (its a very

[Architecture] Siddhi Time Series Extension - Next Steps

Hi, After researching on how to handle seasonality in regression, I have the following findings. 1. Can use dummy variables to capture seasonality. The user needs to add dummy variables to the input stream to capture and quantify seasonality in the regression equation. Therefore, this does not

[Architecture] Siddhi Extensions for Time Series Linear Regression

Waruna and I, have developed two simple siddhi extensions to calculate time series linear regression and outliers. Basically, if a Y event series can be predicted by an X event series, we develop a Y = mx + c , type of equation which quantifies the relationship between X and Y. The actual