Hi Mahesh, Regarding your question:
my outputData Object[]array is in the format of > [mse,beta0,beta1....,betap].But seems to be that cep does not understand it. Did you create an output stream first for the publisher? You need to create a stream with attributes: mse double, beta1 double, ........... and point to that from the publisher. On Wed, Jun 8, 2016 at 1:48 PM, Mahesh Dananjaya <[email protected]> wrote: > Hi Maheshakya, > you can find the details of the queries in this ReadMe [1]. i have add > some changes . so previous querirs may not valid.please use these new > queries in the README. > *1.Streaming Linear regression* > from LinRegInputStream#streaming:streaminglr((learnType), > (batchSize/timeFrame), (numIterations), (stepSize), (miniBatchFraction), > (ci), salary, rbi, walks, strikeouts, errors) > select * > > > > > *insert into regResults; from LinRegInputStream#streaming:streaminglr(0, > 2, 100, 0.00000001, 1, 0.95, salary, rbi, walks, strikeouts, errors)select > *insert into regResults*; > > *2.Streaming KMeans Clustering* > from LinRegInputStream#streaming:streamingkm((learnType), > (batchSize/timeFrame), (numClusters), (numIterations),(alpha), (ci), > salary, rbi, walks, strikeouts, errors) > select * > insert into regResults; > > > > *from > KMeansInputStream#streaming:streamingkm(0,3,0.95,2,10,1,salary,rbi,walks,strikeouts,errors)select > *insert into regResults* > > And i need a help in returning the outputData of my program back to cep. > therefore currenlt you may not find the stream output in event publish.but > you can see the output in the console. i want to understand the final stepd > of putting the output data back to output stream after the batch size is > completed and the algorithms is completed. you may find that following line > passes an exception. Thats have actually no clue of outputData format that > need to give for Output stream. > > Object[] outputData = streamingLinearRegression.regress(eventData); > > > if (outputData == null) { > streamEventChunk.remove(); > } else { > complexEventPopulater.populateComplexEvent(complexEvent, outputData); > } > > my outputData Object[]array is in the format of > [mse,beta0,beta1....,betap].But seems to be that cep does not understand > it. i do it by looking at the time series stream rpocessor extension at > [2].can you please help me with this. > regards, > Mahesh. > > [1] > https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/siddhi/extension/streaming > [2] > https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/LinearRegressionStreamProcessor.java > > On Tue, Jun 7, 2016 at 10:42 PM, Maheshakya Wijewardena < > [email protected]> wrote: > >> Hi Mahesh, >> >> Great work so far. >> >> Regarding the queries: >> >> streamingkm(0, 2,2,20,1,0.95 salary, rbi, walks, strikeouts, errors) >> >> >> Can you give me the definitions of the first few entities in the order. >> Also in previous supervised cases (linear regression), what is the response >> variable, etc. >> I'll go through the code and give you a feedback. >> >> After this, we need to me this implementation into carbon-ml siddhi >> extension. Please also do a similar implementation for logistic regression >> as well because we need to have a streaming version for classification as >> well. >> >> Best regards. >> >> >> >> On Tue, Jun 7, 2016 at 5:50 PM, Mahesh Dananjaya < >> [email protected]> wrote: >> >>> Hi Maheshkya, >>> I have changed the siddhi query for our StreamingKMeansClustering by >>> adding Alpha into the picture which we can use to make data horizon (how >>> quickly a most recent data point becomes a part of the model) and data >>> obsolescence (how long does it take a past data point to become irrelevant >>> to the model)in the streaming clustering algorithms.i have added new >>> changes to repo [1] introducing StreamingKMeansClusteringModel and >>> StreamingKMeansCLustering classes to project.new siddhi query is as follows. >>> >>> from Stream8Input#streaming:streamingkm(0, 2,2,20,1,0.95 salary, rbi, >>> walks, strikeouts, errors) >>> >>> select * >>> insert into regResults; >>> >>> regrads, >>> Mahesh. >>> >>> [1] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc >>> >>> On Mon, Jun 6, 2016 at 6:31 PM, Mahesh Dananjaya < >>> [email protected]> wrote: >>> >>>> Hi Maheshakya, >>>> As we have discussed the architecture of the project i have already >>>> developed a couple of essential components for our project. During last >>>> week i completed the writing cep siddhi extension for our streaming >>>> algorithms which are developed to learn incrementally with past >>>> experiences. I have written the siddhi extensions with StreamProcessor >>>> extension for StreamingLinearRegerssion and StreamingKMeansClustering with >>>> the relevant parameters to call it as siddhi query. On the other hand i did >>>> some research on developing Mini Batch KMeans clustering for our >>>> StreamingKMeansClustering. And also i added the moving window addition to >>>> usual batch processing. And currently i am working on the time based >>>> incremental re-trainign method for siddhi streams. On the >>>> StreamingClustering side i have already part of th >>>> StreamingKMeansClustering with the mini batch KMeans clustering. All the >>>> work i did were pushed to my repo in github [1]. you can find the >>>> development on gsoc/ directory. >>>> And also as the ml team and supun was asked, i have did some timing >>>> and performance analysis for our SGD (Stochastic Gradient Descent) >>>> algorithms for LinearRegression. Those results also add to my repo in [2]. >>>> Now i am developing the rest for our purpose and trying to looked into >>>> other researches on predictive analysis for online big data. Ans also doing >>>> some work related to mini batch KMEans Clustering. And also i have been >>>> working on the performance analysis, accuracy and basic comparison between >>>> mini batch algorithms and moving window algorithms for streaming and >>>> periodic re-training of ML model. thank you. >>>> BR, >>>> Mahesh. >>>> [1] https://github.com/dananjayamahesh/GSOC2016 >>>> [2] >>>> https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/output/lr_timing_10000.jpg >>>> >>>> >>>> On Sat, Jun 4, 2016 at 8:50 PM, Mahesh Dananjaya < >>>> [email protected]> wrote: >>>> >>>>> Hi Maheshkya, >>>>> If you want to run it please use following queries. >>>>> 1. StreamingLInearRegression >>>>> >>>>> from Stream4InputStream#streaming:streaminglr(0, 2, 0.95, salary, rbi, >>>>> walks, strikeouts, errors) >>>>> >>>>> select * >>>>> >>>>> insert into regResults; >>>>> >>>>> from Stream8Input#streaming:streamingkm(0, 2, 0.95,2,20, salary, rbi, >>>>> walks, strikeouts, errors) >>>>> >>>>> select * >>>>> insert into regResults; >>>>> >>>>> in both case the first parameter let you to decide which learning >>>>> methos you want, moving window, batch processing or time based model >>>>> learning. >>>>> BR, >>>>> Mahesh. >>>>> >>>>> On Sat, Jun 4, 2016 at 8:45 PM, Mahesh Dananjaya < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Maheshkaya, >>>>>> I have added the moving window method and update the previos >>>>>> StreamingLinearRegression [1] which only performed batch processing with >>>>>> streaming data. and also i added the StreamingKMeansClustering [1] for >>>>>> our >>>>>> purposes and debugged them.thank you. >>>>>> regards, >>>>>> Mahesh. >>>>>> [1] >>>>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc/siddhi/extension/streaming/src/main/java/org/gsoc/siddhi/extension/streaming >>>>>> >>>>>> On Sat, Jun 4, 2016 at 5:58 PM, Supun Sethunga <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Thanks Mahesh! The graphs look promising! :) >>>>>>> >>>>>>> So by looking at graph, LR with SGD can train a model within 60 >>>>>>> secs (6*10^10 nano sec), using about 900,000 data points . Means, this >>>>>>> online training can handle events/data points coming at rate of 15,000 >>>>>>> per >>>>>>> second (or more) , if the batch size is set to 900,000 (or less) or >>>>>>> window >>>>>>> size is set to 60 secs (or less). This is great IMO! >>>>>>> >>>>>>> On Sat, Jun 4, 2016 at 10:51 AM, Mahesh Dananjaya < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Maheshakya, >>>>>>>> As you requested i can change other parameters as well such as >>>>>>>> feature size(p). Initially i did it with p=3;sure thing. Anyway you >>>>>>>> can see >>>>>>>> and run the code if you want. source is at [1]. the test timing is >>>>>>>> called >>>>>>>> with random data as you requested if you set args[0] to 1. And you can >>>>>>>> find >>>>>>>> the extension and streaming algorithms in gsoc/ directiry[2]. thank >>>>>>>> you. >>>>>>>> BR, >>>>>>>> Mahesh. >>>>>>>> [1] >>>>>>>> https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/src/main/java/org/sparkexample/StreamingLinearRegression.java >>>>>>>> [2] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc >>>>>>>> >>>>>>>> On Sat, Jun 4, 2016 at 10:39 AM, Mahesh Dananjaya < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Hi supun, >>>>>>>>> Though i pushed it yesterday, there was some problems with the >>>>>>>>> network. now you can see them in the repo location [1].I added some >>>>>>>>> Matlab >>>>>>>>> plot you can see the patter there.you can use ml also. Ok sure thing. >>>>>>>>> I can >>>>>>>>> prepare a report or else blog if you want. files are as follows. The >>>>>>>>> y axis >>>>>>>>> is in ns and x axis is in batch size. And also i added two pplots as >>>>>>>>> jpegs[2], so you can easily compare. >>>>>>>>> lr_timing_1000.txt -> batch size incremented by 1000 >>>>>>>>> lr_timing_10000.txt -> batch size incremented by 10000 >>>>>>>>> lr_timing_power10.txt -> batch size incremented by power of 10 >>>>>>>>> >>>>>>>>> In here independent variable is only tha batch size.If you want i >>>>>>>>> can send you making other parameters such as step size, number of >>>>>>>>> iteration, feature vector size as independent variables. please let >>>>>>>>> me know >>>>>>>>> if you want further info. thank you. >>>>>>>>> regards, >>>>>>>>> Mahesh. >>>>>>>>> >>>>>>>>> >>>>>>>>> [1 >>>>>>>>> ]https://github.com/dananjayamahesh/GSOC2016/tree/master/spark-examples/first-example/output >>>>>>>>> [2] >>>>>>>>> https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/output/lr_timing_10000.jpg >>>>>>>>> >>>>>>>>> On Sat, Jun 4, 2016 at 9:58 AM, Supun Sethunga <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Mahesh, >>>>>>>>>> >>>>>>>>>> I have added those timing reports to my repo [1]. >>>>>>>>>> >>>>>>>>>> Whats the file name? :) >>>>>>>>>> >>>>>>>>>> Btw, can you compile simple doc (gdoc) with the above results, >>>>>>>>>> and bring everything to one place? That way it is easy to compare, >>>>>>>>>> and keep >>>>>>>>>> track. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Supun >>>>>>>>>> >>>>>>>>>> On Fri, Jun 3, 2016 at 7:23 PM, Mahesh Dananjaya < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Maheshkya, >>>>>>>>>>> I have added those timing reports to my repo [1].please have a >>>>>>>>>>> look at. three files are there. one is using incremet as 1000 for >>>>>>>>>>> batch >>>>>>>>>>> sizes (lr_timing_1000). Otherone is using incremet by 10000 >>>>>>>>>>> (lr_timing_10000) upto 1 million in both scenarios.you can see the >>>>>>>>>>> reports >>>>>>>>>>> and figures in the location [2] in the repo. i also added the >>>>>>>>>>> streaminglinearregression classes in the repo gsoc folder.thank you. >>>>>>>>>>> regards, >>>>>>>>>>> Mahesh. >>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>> [2] >>>>>>>>>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/spark-examples/first-example/output >>>>>>>>>>> >>>>>>>>>>> On Mon, May 30, 2016 at 9:24 AM, Maheshakya Wijewardena < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>> >>>>>>>>>>>> Thank you for the update. I will look into your implementation. >>>>>>>>>>>> >>>>>>>>>>>> And i will be able to send you the timing/performances analysis >>>>>>>>>>>>> report tomorrow for the SGD functions >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Great. Sent those asap so that we can proceed. >>>>>>>>>>>> >>>>>>>>>>>> Best regards. >>>>>>>>>>>> >>>>>>>>>>>> On Sun, May 29, 2016 at 6:56 PM, Mahesh Dananjaya < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi maheshakay, >>>>>>>>>>>>> I have implemented the linear regression with cep siddhi event >>>>>>>>>>>>> stream with taking batch sizes as parameters from the cep. Now >>>>>>>>>>>>> we can >>>>>>>>>>>>> trying the moving window method to. Before that i think i should >>>>>>>>>>>>> get your >>>>>>>>>>>>> opinion on data structures to save the streaming data.please >>>>>>>>>>>>> check my repo >>>>>>>>>>>>> [1] /gsoc/ folder there you can find all new things i add.. >>>>>>>>>>>>> there in the >>>>>>>>>>>>> extension folder you can find those extension. And i will be able >>>>>>>>>>>>> to send >>>>>>>>>>>>> you the timing/performances analysis report tomorrow for the SGD >>>>>>>>>>>>> functions. >>>>>>>>>>>>> thank you. >>>>>>>>>>>>> regards, >>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, May 27, 2016 at 12:56 PM, Mahesh Dananjaya < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi maheshkaya, >>>>>>>>>>>>>> i have written some siddhi extension and trying to develop a >>>>>>>>>>>>>> one for my one. In time series example in the [1], can you >>>>>>>>>>>>>> please explain >>>>>>>>>>>>>> me the input format and query lines in that example for my >>>>>>>>>>>>>> understanding. >>>>>>>>>>>>>> >>>>>>>>>>>>>> from baseballData#timeseries:regress(2, 10000, 0.95, salary, >>>>>>>>>>>>>> rbi, walks, strikeouts, errors) >>>>>>>>>>>>>> select * >>>>>>>>>>>>>> insert into regResults; >>>>>>>>>>>>>> >>>>>>>>>>>>>> i just want to knwo how i give a set of data into this >>>>>>>>>>>>>> extension and what is baseballData. Is it input stream as >>>>>>>>>>>>>> usual.or any data >>>>>>>>>>>>>> file?how can i find that data set to create dummy input stream >>>>>>>>>>>>>> like >>>>>>>>>>>>>> baseballData? >>>>>>>>>>>>>> >>>>>>>>>>>>>> thank you. >>>>>>>>>>>>>> regards, >>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Thu, May 26, 2016 at 2:58 PM, Mahesh Dananjaya < >>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>> today i got the siddhi and debug the math extention. then >>>>>>>>>>>>>>> did some changes and check. Now i am trying to write same kind >>>>>>>>>>>>>>> of extension >>>>>>>>>>>>>>> in my code base. so i add dependencies and it was built fine. >>>>>>>>>>>>>>> Now i am >>>>>>>>>>>>>>> trying to debug my extension and i did the same thing as i did >>>>>>>>>>>>>>> in previous >>>>>>>>>>>>>>> case. Cep is sending data, bu my extension is not firing in >>>>>>>>>>>>>>> relevant break >>>>>>>>>>>>>>> point. >>>>>>>>>>>>>>> 1. So how can i debug the siddhi extension in my new >>>>>>>>>>>>>>> extension.(you can see it in my example repoo) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I think if i do it correctly we can built the extension for >>>>>>>>>>>>>>> our purpose. And i will send the relevant timing report of SGD >>>>>>>>>>>>>>> algorithms >>>>>>>>>>>>>>> very soon as supun was asking me. thank you. >>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Tue, May 24, 2016 at 11:07 AM, Maheshakya Wijewardena < >>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Also note that there is a calculation interval in the >>>>>>>>>>>>>>>> siddhi time series regression function[1]. You maybe able get >>>>>>>>>>>>>>>> some insight >>>>>>>>>>>>>>>> for this from that as well. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1] https://docs.wso2.com/display/CEP400/Regression >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Tue, May 24, 2016 at 11:03 AM, Maheshakya Wijewardena < >>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> As we discussed offline, we can use similar mechanism to >>>>>>>>>>>>>>>>> train linear regression models, logistic regression models >>>>>>>>>>>>>>>>> and k-means >>>>>>>>>>>>>>>>> clustering models. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It is very interesting that i have found that somethings >>>>>>>>>>>>>>>>>> that can make use of our work. In the cep 4.0 documentation >>>>>>>>>>>>>>>>>> there is a >>>>>>>>>>>>>>>>>> Custom Stream Processor Extention program [1]. There is a >>>>>>>>>>>>>>>>>> example of >>>>>>>>>>>>>>>>>> LinearRegressionStreamProcessor [1]. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> As we have to train predictive models with Spark, you can >>>>>>>>>>>>>>>>> write wrappers around regression/clustering models of Spark. >>>>>>>>>>>>>>>>> Refer to >>>>>>>>>>>>>>>>> Siddhi time series regression source codes[1][2]. You can >>>>>>>>>>>>>>>>> write a streaming >>>>>>>>>>>>>>>>> linear regression class for ML in a similar fashion by >>>>>>>>>>>>>>>>> wrapping Spark mllib >>>>>>>>>>>>>>>>> implementations. You can use the methods "addEvent", >>>>>>>>>>>>>>>>> "removeEvent", etc. >>>>>>>>>>>>>>>>> (may have to be changed according to requirements) for the >>>>>>>>>>>>>>>>> similar purpose. >>>>>>>>>>>>>>>>> You can introduce >>>>>>>>>>>>>>>>> trainLinearRegression/LogisticRegression/Kmeans which >>>>>>>>>>>>>>>>> does a similar thing as in createLinearRegression in those >>>>>>>>>>>>>>>>> time series >>>>>>>>>>>>>>>>> functions. In the processData method you can use Spark mllib >>>>>>>>>>>>>>>>> classes to >>>>>>>>>>>>>>>>> actually train models and return the model weights, >>>>>>>>>>>>>>>>> evaluation metrics. So, >>>>>>>>>>>>>>>>> converting streams into RDDs and retrieving information from >>>>>>>>>>>>>>>>> the trained >>>>>>>>>>>>>>>>> models shall happen in this method. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In the stream processor extension example, you can >>>>>>>>>>>>>>>>> retrieve those values then use them to train new models with >>>>>>>>>>>>>>>>> new batches. >>>>>>>>>>>>>>>>> Weights/cluster centers maybe passed as initialization >>>>>>>>>>>>>>>>> parameters for the >>>>>>>>>>>>>>>>> wrappers. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please note that we have to figure out the best siddhi >>>>>>>>>>>>>>>>> extension type for this process. In the siddhi query, we >>>>>>>>>>>>>>>>> define batch size, >>>>>>>>>>>>>>>>> type of algorithm and number of features (there can be more). >>>>>>>>>>>>>>>>> After batch >>>>>>>>>>>>>>>>> size number of events received, train a model and save >>>>>>>>>>>>>>>>> parameters, return >>>>>>>>>>>>>>>>> evaluation metric. With the next batch, retrain the model >>>>>>>>>>>>>>>>> initialized with >>>>>>>>>>>>>>>>> previously learned parameters. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We also may need to test the same scenario with a moving >>>>>>>>>>>>>>>>> window, but I suspect that that approach may become so slow >>>>>>>>>>>>>>>>> as a model is >>>>>>>>>>>>>>>>> trained each time an event is received. So, we may have to >>>>>>>>>>>>>>>>> change the >>>>>>>>>>>>>>>>> number of slots the moving window moves at a time (eg: not >>>>>>>>>>>>>>>>> one by one, but >>>>>>>>>>>>>>>>> ten by ten). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Once this is resolved, majority of the research part will >>>>>>>>>>>>>>>>> be finished and all we will be left to do is implementing >>>>>>>>>>>>>>>>> wrappers around >>>>>>>>>>>>>>>>> the 3 learning algorithms we consider. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>> https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/linreg/RegressionCalculator.java >>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>> https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/linreg/SimpleLinearRegressionCalculator.java >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Sat, May 21, 2016 at 2:55 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Maheshkya, >>>>>>>>>>>>>>>>>> shall we use [1] for our work? i am checking the >>>>>>>>>>>>>>>>>> possibility. >>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Inbuilt+Windows#InbuiltWindows-lengthlength >>>>>>>>>>>>>>>>>> [3] >>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Aggregate+Function >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Sat, May 21, 2016 at 2:44 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>> It is very interesting that i have found that somethings >>>>>>>>>>>>>>>>>>> that can make use of our work. In the cep 4.0 documentation >>>>>>>>>>>>>>>>>>> there is a >>>>>>>>>>>>>>>>>>> Custom Stream Processor Extention program [1]. There is a >>>>>>>>>>>>>>>>>>> example of >>>>>>>>>>>>>>>>>>> LinearRegressionStreamProcessor [1] and also i saw >>>>>>>>>>>>>>>>>>> private int batchSize = 1000000000; i am going through >>>>>>>>>>>>>>>>>>> this one. >>>>>>>>>>>>>>>>>>> Please check whether we can use. WIll there be any >>>>>>>>>>>>>>>>>>> compatibility or support issue? >>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Sat, May 21, 2016 at 11:52 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi maheshakya, >>>>>>>>>>>>>>>>>>>> anyway how can test any siddhi extention after write it >>>>>>>>>>>>>>>>>>>> without integrating it to cep.can you please explain me >>>>>>>>>>>>>>>>>>>> the procedure. i am >>>>>>>>>>>>>>>>>>>> referring to [1] [2] [3] [4]. thank you. >>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+Extensions+to+Siddhi >>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+a+Custom+Function >>>>>>>>>>>>>>>>>>>> [3] >>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+a+Custom+Window >>>>>>>>>>>>>>>>>>>> [4] >>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+Extensions+to+Siddhi >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Thu, May 19, 2016 at 12:08 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>> thank you for the feedback. I have add data-sets into >>>>>>>>>>>>>>>>>>>>> repo. data-sets/lr. I am all right with next week.Now i >>>>>>>>>>>>>>>>>>>>> am writing some >>>>>>>>>>>>>>>>>>>>> examples to collect samples and build mini batches and >>>>>>>>>>>>>>>>>>>>> run the algorithms >>>>>>>>>>>>>>>>>>>>> on those mini-batches. thank you. will add those into >>>>>>>>>>>>>>>>>>>>> repo soon.I am still >>>>>>>>>>>>>>>>>>>>> working on that siddhi extention.i will let you know the >>>>>>>>>>>>>>>>>>>>> progress. >>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>> mahesh. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Thu, May 19, 2016 at 11:10 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>> Wijewardena <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I've look into your code sample of streaming linear >>>>>>>>>>>>>>>>>>>>>> regression. Looks good to me, apart from few issues in >>>>>>>>>>>>>>>>>>>>>> coding practices >>>>>>>>>>>>>>>>>>>>>> which we can improve when you're doing the >>>>>>>>>>>>>>>>>>>>>> implementations in carbon-ml and >>>>>>>>>>>>>>>>>>>>>> during the code reviews. You are using a set of files as >>>>>>>>>>>>>>>>>>>>>> mini-batches of >>>>>>>>>>>>>>>>>>>>>> data, right? Can you also send us the datasets you've >>>>>>>>>>>>>>>>>>>>>> been using. I'd like >>>>>>>>>>>>>>>>>>>>>> to run this. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> does that cep problem is now all right that we were >>>>>>>>>>>>>>>>>>>>>>> trying to fix. I am still using those pre-build >>>>>>>>>>>>>>>>>>>>>>> versions. If so i can merge >>>>>>>>>>>>>>>>>>>>>>> with the latest one. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> I'll check this and let you know. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Can we arrange a meeting (preferably in WSO2 offices) >>>>>>>>>>>>>>>>>>>>>> in next week with ML team members as well. Coding period >>>>>>>>>>>>>>>>>>>>>> begins on next >>>>>>>>>>>>>>>>>>>>>> Monday, so it's better to get overall feedback from >>>>>>>>>>>>>>>>>>>>>> others and discuss more >>>>>>>>>>>>>>>>>>>>>> about the project. Let me know convenient time slots for >>>>>>>>>>>>>>>>>>>>>> you. I'll arrange >>>>>>>>>>>>>>>>>>>>>> a meeting with ML team. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:53 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>> Ok. I will check it.you have sent me those relevant >>>>>>>>>>>>>>>>>>>>>>> references and i am working on that thing.thank you. >>>>>>>>>>>>>>>>>>>>>>> does that cep problem >>>>>>>>>>>>>>>>>>>>>>> is now all right that we were trying to fix. I am still >>>>>>>>>>>>>>>>>>>>>>> using those >>>>>>>>>>>>>>>>>>>>>>> pre-build versions. If so i can merge with the latest >>>>>>>>>>>>>>>>>>>>>>> one.thanks. >>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:44 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>> Wijewardena <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> You don't actually have to implement anything in >>>>>>>>>>>>>>>>>>>>>>>> spark streaming. Try to understand how streaming data >>>>>>>>>>>>>>>>>>>>>>>> is handled in and the >>>>>>>>>>>>>>>>>>>>>>>> specifics of the underlying algorithms in streaming. >>>>>>>>>>>>>>>>>>>>>>>> What we want to do is having the similar algorithms >>>>>>>>>>>>>>>>>>>>>>>> that support CEP event streams with siddhi. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:38 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>> Did you check the repo. I will add recent works >>>>>>>>>>>>>>>>>>>>>>>>> today.And also i was going through the Java docs >>>>>>>>>>>>>>>>>>>>>>>>> related to spark streaming >>>>>>>>>>>>>>>>>>>>>>>>> work. It is with that scala API. thank you. >>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 17, 2016 at 10:11 AM, Mahesh Dananjaya >>>>>>>>>>>>>>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>> I have gone through the Java Docs and run some of >>>>>>>>>>>>>>>>>>>>>>>>>> the Spark examples on spark shell which are >>>>>>>>>>>>>>>>>>>>>>>>>> paramount improtant for our >>>>>>>>>>>>>>>>>>>>>>>>>> work. Then i have been writing my codes to check the >>>>>>>>>>>>>>>>>>>>>>>>>> Linear regression, K >>>>>>>>>>>>>>>>>>>>>>>>>> means for streaming. please check my git repo [1]. I >>>>>>>>>>>>>>>>>>>>>>>>>> think now i have to >>>>>>>>>>>>>>>>>>>>>>>>>> ask on dev regarding the capturing event streams for >>>>>>>>>>>>>>>>>>>>>>>>>> our work. I will >>>>>>>>>>>>>>>>>>>>>>>>>> update the recent things on git. check the >>>>>>>>>>>>>>>>>>>>>>>>>> park-example directory for java. >>>>>>>>>>>>>>>>>>>>>>>>>> examples run on git shell is not included there. In >>>>>>>>>>>>>>>>>>>>>>>>>> my case i think i have >>>>>>>>>>>>>>>>>>>>>>>>>> to build mini batches from data streams that comes >>>>>>>>>>>>>>>>>>>>>>>>>> as individual samples. >>>>>>>>>>>>>>>>>>>>>>>>>> Now i am working on some coding to collect mini >>>>>>>>>>>>>>>>>>>>>>>>>> batches from data >>>>>>>>>>>>>>>>>>>>>>>>>> streams.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 17, 2016 at 10:10 AM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>> I have gone through the Java Docs and run some >>>>>>>>>>>>>>>>>>>>>>>>>>> of the Spark examples on spark shell which are >>>>>>>>>>>>>>>>>>>>>>>>>>> paramount improtant for our >>>>>>>>>>>>>>>>>>>>>>>>>>> work. Then i have been writing my codes to check >>>>>>>>>>>>>>>>>>>>>>>>>>> the Linear regression, K >>>>>>>>>>>>>>>>>>>>>>>>>>> means for streaming. please check my git repo [1]. >>>>>>>>>>>>>>>>>>>>>>>>>>> I think now i have to >>>>>>>>>>>>>>>>>>>>>>>>>>> ask on dev regarding the capturing event streams >>>>>>>>>>>>>>>>>>>>>>>>>>> for our work. I will >>>>>>>>>>>>>>>>>>>>>>>>>>> update the recent things on git. check the >>>>>>>>>>>>>>>>>>>>>>>>>>> park-example directory for java. >>>>>>>>>>>>>>>>>>>>>>>>>>> examples run on git shell is not included there. In >>>>>>>>>>>>>>>>>>>>>>>>>>> my case i think i have >>>>>>>>>>>>>>>>>>>>>>>>>>> to build mini batches from data streams that comes >>>>>>>>>>>>>>>>>>>>>>>>>>> as individual samples. >>>>>>>>>>>>>>>>>>>>>>>>>>> Now i am working on some coding to collect mini >>>>>>>>>>>>>>>>>>>>>>>>>>> batches from data >>>>>>>>>>>>>>>>>>>>>>>>>>> streams.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 16, 2016 at 1:19 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>> thank you. i will update the repo today.thank >>>>>>>>>>>>>>>>>>>>>>>>>>>> you.i changed the carbon ml siddhi extention and >>>>>>>>>>>>>>>>>>>>>>>>>>>> see how the changes are >>>>>>>>>>>>>>>>>>>>>>>>>>>> effecting. i will update the progress as soon as >>>>>>>>>>>>>>>>>>>>>>>>>>>> possible.thank you. i had >>>>>>>>>>>>>>>>>>>>>>>>>>>> some problem in spark mllib dependency. i was >>>>>>>>>>>>>>>>>>>>>>>>>>>> fixing that. >>>>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>> p.s: do i need to maintain a blog? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 16, 2016 at 10:02 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sorry for replying late. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thank you for the update. I believe you have >>>>>>>>>>>>>>>>>>>>>>>>>>>>> done some implementations with with Spark MLLIb >>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms in streaming >>>>>>>>>>>>>>>>>>>>>>>>>>>>> fashion as we have discussed. If so, can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>> please share your code in a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Github repo. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Now i want to implements some machine learning >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms with importing mllib and want to run >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> within your code base >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> For the moment you can try out editing the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> same class PredictStreamProcessor in the siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extension in carbon-ml. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Later we will add this separately. You should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>> able to add >>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.apache.spark.mllib. classes to there. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> And i want to see how event streams are coming >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from cep. As i think it is not in a RDD format >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> since it is arriving as the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> individual samples. I will send a email to dev >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> asking about how to get the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streams. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please pay attention to length[1] and >>>>>>>>>>>>>>>>>>>>>>>>>>>>> lengthbatch[1] inbuilt windows in siddhi. What >>>>>>>>>>>>>>>>>>>>>>>>>>>>> you need to write are >>>>>>>>>>>>>>>>>>>>>>>>>>>>> functions similar to a custom aggregate >>>>>>>>>>>>>>>>>>>>>>>>>>>>> function[2]. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> When you send the email to dev list, explain >>>>>>>>>>>>>>>>>>>>>>>>>>>>> your requirement. You need to get a set of event >>>>>>>>>>>>>>>>>>>>>>>>>>>>> with from a stream with a >>>>>>>>>>>>>>>>>>>>>>>>>>>>> specified window size (number of events). Then >>>>>>>>>>>>>>>>>>>>>>>>>>>>> build a model within that >>>>>>>>>>>>>>>>>>>>>>>>>>>>> function. You also need to retain the data >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (learned weights, cluster >>>>>>>>>>>>>>>>>>>>>>>>>>>>> centers, etc.) from the previous window to use in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> the current window. Ask >>>>>>>>>>>>>>>>>>>>>>>>>>>>> what can be the most suitable option for this >>>>>>>>>>>>>>>>>>>>>>>>>>>>> among the set of siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extensions given. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Inbuilt+Windows#InbuiltWindows-lengthlength >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Aggregate+Function >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 11, 2016 at 1:43 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> From: Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Date: Wed, May 11, 2016 at 1:43 PM >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Subject: Re: [Dev] GSOC2016: [ML][CEP] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Predictive analytic with online data for WSO2 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Machine Learner >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> To: Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sorry for not updating. I did what you wanted >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> me to do. I checked the code base and train >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functions. I went through those >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java docs. I went through the carbon-ml current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation of LG and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> K-Mean. And i had Apache Spark and i tried with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> several examples. Now i >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> want to implements some machine learning >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms with importing mllib >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and want to run within your code base. Can you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> help me with that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And i want to see how event streams are >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> coming from cep. As i think it is not in a RDD >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> format since it is arriving >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as the individual samples. I will send a email >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to dev asking about how to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get the streams. I debugged many of those >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> functions in the code base. So >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need further instructions to proceed.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 11, 2016 at 10:32 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any update on your progress? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 4, 2016 at 8:35 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <[email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is that "Put break points in train methods >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in Linear Regression class" means the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LinearRegrassion.java class in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.core? is that the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct file? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, this is the correct place. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can refer to spark programming >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> guide[1][2] as well as our ML code base when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you try those algorithms out. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Please try to do rough implementations of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming versions of linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regression, logistic regression and k-means >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> clustering as we have discussed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the proposal in plain Java. It's better if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you can create a git repo and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> share your code once you have made some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> progress. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Were you able debug and understand the flow >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of the ML siddhi extension? I hope you haven't >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> encountered more errors >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> after switching the released version of CEP. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is this Friday okay for you? Afternoon at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2:00 pm? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://spark.apache.org/docs/latest/programming-guide.html >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://spark.apache.org/docs/latest/mllib-guide.html >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 4, 2016 at 1:07 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <[email protected]> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have been looking into some algorithms >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> related to stochastic gradient descent based >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms.anything i should >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> focus please let me know.Ans also i will be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> available for calling this week >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and next week.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 5:05 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <[email protected]> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thank you.that's good. i have been trying >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to fix that for couple of days. please >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> inform me when it will be fixed.now >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> i have been testing the ML algorithms and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying to identify the flow and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the hierarchy. is that "Put break points in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> train methods in Linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regression class" means the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/ LinearRegrassion.java class >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the org.wso2.carbon.ml.core? is that the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> correct file? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am planning to write some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> programs to use apache spark mllib >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms. and i refer to [1] and some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wso2 documentations to get some idea about >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ML structure.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1]nirmalfdo.blogspot.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 4:36 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have checked. It seems the issue you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have encountered is cause only in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current development branch of the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-cep. It doesn't identify the ML >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> siddhi extension as an extension. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ML siddhi extension works fine in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest release of CEP (4.1.0) [1]. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Until we figure out the reason and come >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> up with a solution, can you use the latest >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CEP release for your work. It's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fine to use that since you haven't started >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> actual development yet. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://wso2.com/products/complex-event-processor/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 3:19 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is is vital to use those local repo in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> my upcoming implementation? >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes. The remote p2-repo contains the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> p2-repos of released versions. What you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have to develop on is the current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> master of the carbon-ml and product-ml. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You can try out with the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> modification I have suggested. In the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> meantime, I'll verify whether the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> current repos are working as expected. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am trying to debug the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml org.wso2.carbon.ml.core by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> putting some break point in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/Linear Regression >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It's great that you have started >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> looking at the implementation of linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regression as well. Put break >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> points in train methods in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> LinearRegression class. This is being used >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you run linear regression from UI. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I can see some comments left behind for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming algo as well.thank you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You may be referring to the linear >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regression with SGD model. Here, there's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> no retraining with streaming data >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> involved. The SGD with minibatches is used >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to train the model with the data >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> set only once. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What you have to do is create a similar >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism to involve streaming data and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> retrain models. We will get to that >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> part once you get comfortable with siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extensions. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BTW, is it possible for you to join a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> call on this Friday or in the next week. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We'll try to resolve your current >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues and discuss further on project. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 1:03 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <[email protected]> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok to go with p2 repo at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://product-dist.wso2.com/p2/carbon/releases/wilkes/features/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than the P2-repo at >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml/modules/p2-profile/target/p2-repo >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> local repo.What is the impact?.Is is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vital to use those local repo in my >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> upcoming implementation?so i was trying >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to give remote p2 repo to cep built >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> by source and trying to debug the cep ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> extension and got the same error >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as yesterday. But pre-built product is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> working fine. Therefore now i am >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying as you described in the last email. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am trying to debug the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml org.wso2.carbon.ml.core by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> putting some break point in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/Linear Regression. I am >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying to trigger it with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml project with data set. Does >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that Linear Regression also in UI is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> consuming that spark algorithms or is it >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in another place? I can see some >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments left behind for streaming algo >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as well.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 9:35 AM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The earlier error you have mentioned >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may occur due to the incompatible Siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> versions in ML p2-repo and CEP, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> when you add the p2-repo from prodcut-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you built. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Current siddhi version in product-cep >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is 3.0.6-SNAPSHOT[1], but in ML it's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3.0.2. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you try changing the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> siddhi.version in carbon-ml/pom.xml to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3.0.6-SNAPSHOT, build carbon-ml, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> then build product-ml again. After this, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> add p2-repo as a local repository >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again to fresh CEP pack and try it out. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 2, 2016 at 7:02 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya,now i remotely debug >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the CEP extension for ML Prediction. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> What i did was, i have all the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-build version of CEP and ML. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Therefore i used that built CEP and did >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the same thing that i was doing with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the source code. I think the only >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> change i did was install those packages >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from remote p2 repo. This was work >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fine and i debugged the carbon-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.siddhi.extension as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> described in the [1]. So now i have to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> try same thing with the build by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> source content.thank you.[1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-SiddhisyntaxfortheextensionBR >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> , >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 5:49 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya Wijewardena < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh,Congratulations and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> welcome to GSoC 2016. You did a great >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> job in preparing the proposal. Now >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> it's time to dig deep and get started >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with the project.First of all you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need to familiarize with the code >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> base. We have agreed to implement this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with CEP event streams. We already >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have a CEP extension for predictions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1][2]. Go through this implementation >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and familiarize your self with that. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You need to understand how:Even >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streams are consumedpredictions are >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> made >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from individual eventResults are sent >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> backGet WSO2 ML and CEP sources (You >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> may use latest released version of >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> CEP) and build the products. Get both >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml[3] and product-ml[4] masters >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and create new branches for your >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work from masters.After you build the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> products, you may need to do remote >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> debugging[5] to understand the flow. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So please follow an example of real >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> time prediction with ML with debugging >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and get some idea. The component you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need to debug is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.siddhi.extension.Next >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tasks would be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementing online learning >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms in plain java with spark ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lib and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integrating those to ML. We also need >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to come up with a proper and detailed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> architecture to employ those >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> algorithms in ML. Getting familiar >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> aforementioned sections would give you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some insight on how this should be >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implemented. So please try to get a >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quick grasp then you can start the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation. Let us know if you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> have any questions or you get stuck >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> somewhere. Also, please always add >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> WSO2 developer's list as well when you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> communicate with us regarding the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> project so that you can get opinions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feedback from others as well.Best >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards.[1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension[2] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/extensions/org.wso2.carbon.ml.siddhi.extension[3] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/wso2/carbon-ml[4] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/wso2/product-ml[5] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://dzone.com/articles/how-debug-wso2-carbon-kernel >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 3:33 PM, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote:Hi,thank you for accepting my >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> GSOC 2016 proposal and i am looking >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> forward for the further instruction >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and project continuation. thank you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> very much.regards,Mahesh.-- Pruthuvi >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected]+94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>> [email protected] >>>>>>>>>>>> +94711228855 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Supun Sethunga* >>>>>>>>>> Senior Software Engineer >>>>>>>>>> WSO2, Inc. >>>>>>>>>> http://wso2.com/ >>>>>>>>>> lean | enterprise | middleware >>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Supun Sethunga* >>>>>>> Senior Software Engineer >>>>>>> WSO2, Inc. >>>>>>> http://wso2.com/ >>>>>>> lean | enterprise | middleware >>>>>>> Mobile : +94 716546324 >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> Pruthuvi Maheshakya Wijewardena >> [email protected] >> +94711228855 >> >> >> > -- Pruthuvi Maheshakya Wijewardena [email protected] +94711228855
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
