Hi Maheshakya, As you requested i can change other parameters as well such as feature size(p). Initially i did it with p=3;sure thing. Anyway you can see and run the code if you want. source is at [1]. the test timing is called with random data as you requested if you set args[0] to 1. And you can find the extension and streaming algorithms in gsoc/ directiry[2]. thank you. BR, Mahesh. [1] https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/src/main/java/org/sparkexample/StreamingLinearRegression.java [2] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc
On Sat, Jun 4, 2016 at 10:39 AM, Mahesh Dananjaya <dananjayamah...@gmail.com > wrote: > Hi supun, > Though i pushed it yesterday, there was some problems with the network. > now you can see them in the repo location [1].I added some Matlab plot you > can see the patter there.you can use ml also. Ok sure thing. I can prepare > a report or else blog if you want. files are as follows. The y axis is in > ns and x axis is in batch size. And also i added two pplots as jpegs[2], so > you can easily compare. > lr_timing_1000.txt -> batch size incremented by 1000 > lr_timing_10000.txt -> batch size incremented by 10000 > lr_timing_power10.txt -> batch size incremented by power of 10 > > In here independent variable is only tha batch size.If you want i can send > you making other parameters such as step size, number of iteration, feature > vector size as independent variables. please let me know if you want > further info. thank you. > regards, > Mahesh. > > > [1 > ]https://github.com/dananjayamahesh/GSOC2016/tree/master/spark-examples/first-example/output > [2] > https://github.com/dananjayamahesh/GSOC2016/blob/master/spark-examples/first-example/output/lr_timing_10000.jpg > > On Sat, Jun 4, 2016 at 9:58 AM, Supun Sethunga <sup...@wso2.com> wrote: > >> Hi Mahesh, >> >> I have added those timing reports to my repo [1]. >> >> Whats the file name? :) >> >> Btw, can you compile simple doc (gdoc) with the above results, and bring >> everything to one place? That way it is easy to compare, and keep track. >> >> Thanks, >> Supun >> >> On Fri, Jun 3, 2016 at 7:23 PM, Mahesh Dananjaya < >> dananjayamah...@gmail.com> wrote: >> >>> Hi Maheshkya, >>> I have added those timing reports to my repo [1].please have a look at. >>> three files are there. one is using incremet as 1000 for batch sizes >>> (lr_timing_1000). Otherone is using incremet by 10000 (lr_timing_10000) >>> upto 1 million in both scenarios.you can see the reports and figures in the >>> location [2] in the repo. i also added the streaminglinearregression >>> classes in the repo gsoc folder.thank you. >>> regards, >>> Mahesh. >>> [1]https://github.com/dananjayamahesh/GSOC2016 >>> [2] >>> https://github.com/dananjayamahesh/GSOC2016/tree/master/spark-examples/first-example/output >>> >>> On Mon, May 30, 2016 at 9:24 AM, Maheshakya Wijewardena < >>> mahesha...@wso2.com> wrote: >>> >>>> Hi Mahesh, >>>> >>>> Thank you for the update. I will look into your implementation. >>>> >>>> And i will be able to send you the timing/performances analysis report >>>>> tomorrow for the SGD functions >>>>> >>>> >>>> Great. Sent those asap so that we can proceed. >>>> >>>> Best regards. >>>> >>>> On Sun, May 29, 2016 at 6:56 PM, Mahesh Dananjaya < >>>> dananjayamah...@gmail.com> wrote: >>>> >>>>> >>>>> Hi maheshakay, >>>>> I have implemented the linear regression with cep siddhi event stream >>>>> with taking batch sizes as parameters from the cep. Now we can trying the >>>>> moving window method to. Before that i think i should get your opinion on >>>>> data structures to save the streaming data.please check my repo [1] >>>>> /gsoc/ >>>>> folder there you can find all new things i add.. there in the extension >>>>> folder you can find those extension. And i will be able to send you the >>>>> timing/performances analysis report tomorrow for the SGD functions. thank >>>>> you. >>>>> regards, >>>>> Mahesh. >>>>> [1] https://github.com/dananjayamahesh/GSOC2016/tree/master/gsoc >>>>> >>>>> >>>>> On Fri, May 27, 2016 at 12:56 PM, Mahesh Dananjaya < >>>>> dananjayamah...@gmail.com> wrote: >>>>> >>>>>> Hi maheshkaya, >>>>>> i have written some siddhi extension and trying to develop a one for >>>>>> my one. In time series example in the [1], can you please explain me the >>>>>> input format and query lines in that example for my understanding. >>>>>> >>>>>> from baseballData#timeseries:regress(2, 10000, 0.95, salary, rbi, >>>>>> walks, strikeouts, errors) >>>>>> select * >>>>>> insert into regResults; >>>>>> >>>>>> i just want to knwo how i give a set of data into this extension and >>>>>> what is baseballData. Is it input stream as usual.or any data file?how >>>>>> can >>>>>> i find that data set to create dummy input stream like baseballData? >>>>>> >>>>>> thank you. >>>>>> regards, >>>>>> Mahesh. >>>>>> [1] >>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>> >>>>>> On Thu, May 26, 2016 at 2:58 PM, Mahesh Dananjaya < >>>>>> dananjayamah...@gmail.com> wrote: >>>>>> >>>>>>> Hi Maheshakya, >>>>>>> today i got the siddhi and debug the math extention. then did some >>>>>>> changes and check. Now i am trying to write same kind of extension in my >>>>>>> code base. so i add dependencies and it was built fine. Now i am trying >>>>>>> to >>>>>>> debug my extension and i did the same thing as i did in previous case. >>>>>>> Cep >>>>>>> is sending data, bu my extension is not firing in relevant break point. >>>>>>> 1. So how can i debug the siddhi extension in my new extension.(you >>>>>>> can see it in my example repoo) >>>>>>> >>>>>>> I think if i do it correctly we can built the extension for our >>>>>>> purpose. And i will send the relevant timing report of SGD algorithms >>>>>>> very >>>>>>> soon as supun was asking me. thank you. >>>>>>> regards, >>>>>>> Mahesh. >>>>>>> >>>>>>> On Tue, May 24, 2016 at 11:07 AM, Maheshakya Wijewardena < >>>>>>> mahesha...@wso2.com> wrote: >>>>>>> >>>>>>>> Also note that there is a calculation interval in the siddhi time >>>>>>>> series regression function[1]. You maybe able get some insight for this >>>>>>>> from that as well. >>>>>>>> >>>>>>>> [1] https://docs.wso2.com/display/CEP400/Regression >>>>>>>> >>>>>>>> On Tue, May 24, 2016 at 11:03 AM, Maheshakya Wijewardena < >>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>> >>>>>>>>> Hi Mahesh, >>>>>>>>> >>>>>>>>> As we discussed offline, we can use similar mechanism to train >>>>>>>>> linear regression models, logistic regression models and k-means >>>>>>>>> clustering >>>>>>>>> models. >>>>>>>>> >>>>>>>>> It is very interesting that i have found that somethings that can >>>>>>>>>> make use of our work. In the cep 4.0 documentation there is a Custom >>>>>>>>>> Stream >>>>>>>>>> Processor Extention program [1]. There is a example of >>>>>>>>>> LinearRegressionStreamProcessor [1]. >>>>>>>>>> >>>>>>>>> >>>>>>>>> As we have to train predictive models with Spark, you can write >>>>>>>>> wrappers around regression/clustering models of Spark. Refer to >>>>>>>>> Siddhi time >>>>>>>>> series regression source codes[1][2]. You can write a streaming linear >>>>>>>>> regression class for ML in a similar fashion by wrapping Spark mllib >>>>>>>>> implementations. You can use the methods "addEvent", "removeEvent", >>>>>>>>> etc. >>>>>>>>> (may have to be changed according to requirements) for the similar >>>>>>>>> purpose. >>>>>>>>> You can introduce trainLinearRegression/LogisticRegression/Kmeans >>>>>>>>> which >>>>>>>>> does a similar thing as in createLinearRegression in those time series >>>>>>>>> functions. In the processData method you can use Spark mllib classes >>>>>>>>> to >>>>>>>>> actually train models and return the model weights, evaluation >>>>>>>>> metrics. So, >>>>>>>>> converting streams into RDDs and retrieving information from the >>>>>>>>> trained >>>>>>>>> models shall happen in this method. >>>>>>>>> >>>>>>>>> In the stream processor extension example, you can retrieve those >>>>>>>>> values then use them to train new models with new batches. >>>>>>>>> Weights/cluster >>>>>>>>> centers maybe passed as initialization parameters for the wrappers. >>>>>>>>> >>>>>>>>> Please note that we have to figure out the best siddhi extension >>>>>>>>> type for this process. In the siddhi query, we define batch size, >>>>>>>>> type of >>>>>>>>> algorithm and number of features (there can be more). After batch size >>>>>>>>> number of events received, train a model and save parameters, return >>>>>>>>> evaluation metric. With the next batch, retrain the model initialized >>>>>>>>> with >>>>>>>>> previously learned parameters. >>>>>>>>> >>>>>>>>> We also may need to test the same scenario with a moving window, >>>>>>>>> but I suspect that that approach may become so slow as a model is >>>>>>>>> trained >>>>>>>>> each time an event is received. So, we may have to change the number >>>>>>>>> of >>>>>>>>> slots the moving window moves at a time (eg: not one by one, but ten >>>>>>>>> by >>>>>>>>> ten). >>>>>>>>> >>>>>>>>> Once this is resolved, majority of the research part will be >>>>>>>>> finished and all we will be left to do is implementing wrappers >>>>>>>>> around the >>>>>>>>> 3 learning algorithms we consider. >>>>>>>>> >>>>>>>>> Best regards. >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/linreg/RegressionCalculator.java >>>>>>>>> [2] >>>>>>>>> https://github.com/wso2/siddhi/blob/master/modules/siddhi-extensions/timeseries/src/main/java/org/wso2/siddhi/extension/timeseries/linreg/SimpleLinearRegressionCalculator.java >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, May 21, 2016 at 2:55 PM, Mahesh Dananjaya < >>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi Maheshkya, >>>>>>>>>> shall we use [1] for our work? i am checking the possibility. >>>>>>>>>> BR, >>>>>>>>>> Mahesh. >>>>>>>>>> [1] >>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>> [2] >>>>>>>>>> https://docs.wso2.com/display/CEP400/Inbuilt+Windows#InbuiltWindows-lengthlength >>>>>>>>>> [3] >>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Aggregate+Function >>>>>>>>>> >>>>>>>>>> On Sat, May 21, 2016 at 2:44 PM, Mahesh Dananjaya < >>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>> It is very interesting that i have found that somethings that >>>>>>>>>>> can make use of our work. In the cep 4.0 documentation there is a >>>>>>>>>>> Custom >>>>>>>>>>> Stream Processor Extention program [1]. There is a example of >>>>>>>>>>> LinearRegressionStreamProcessor [1] and also i saw >>>>>>>>>>> private int batchSize = 1000000000; i am going through this >>>>>>>>>>> one. >>>>>>>>>>> Please check whether we can use. WIll there be any compatibility >>>>>>>>>>> or support issue? >>>>>>>>>>> regards, >>>>>>>>>>> Mahesh. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> [1] >>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Stream+Processor+Extension >>>>>>>>>>> >>>>>>>>>>> On Sat, May 21, 2016 at 11:52 AM, Mahesh Dananjaya < >>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi maheshakya, >>>>>>>>>>>> anyway how can test any siddhi extention after write it without >>>>>>>>>>>> integrating it to cep.can you please explain me the procedure. i am >>>>>>>>>>>> referring to [1] [2] [3] [4]. thank you. >>>>>>>>>>>> BR, >>>>>>>>>>>> Mahesh. >>>>>>>>>>>> >>>>>>>>>>>> [1] >>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+Extensions+to+Siddhi >>>>>>>>>>>> [2] >>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+a+Custom+Function >>>>>>>>>>>> [3] >>>>>>>>>>>> https://docs.wso2.com/display/CEP310/Writing+a+Custom+Window >>>>>>>>>>>> [4] >>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+Extensions+to+Siddhi >>>>>>>>>>>> >>>>>>>>>>>> On Thu, May 19, 2016 at 12:08 PM, Mahesh Dananjaya < >>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>> thank you for the feedback. I have add data-sets into repo. >>>>>>>>>>>>> data-sets/lr. I am all right with next week.Now i am writing some >>>>>>>>>>>>> examples >>>>>>>>>>>>> to collect samples and build mini batches and run the algorithms >>>>>>>>>>>>> on those >>>>>>>>>>>>> mini-batches. thank you. will add those into repo soon.I am still >>>>>>>>>>>>> working >>>>>>>>>>>>> on that siddhi extention.i will let you know the progress. >>>>>>>>>>>>> BR, >>>>>>>>>>>>> mahesh. >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, May 19, 2016 at 11:10 AM, Maheshakya Wijewardena < >>>>>>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I've look into your code sample of streaming linear >>>>>>>>>>>>>> regression. Looks good to me, apart from few issues in coding >>>>>>>>>>>>>> practices >>>>>>>>>>>>>> which we can improve when you're doing the implementations in >>>>>>>>>>>>>> carbon-ml and >>>>>>>>>>>>>> during the code reviews. You are using a set of files as >>>>>>>>>>>>>> mini-batches of >>>>>>>>>>>>>> data, right? Can you also send us the datasets you've been >>>>>>>>>>>>>> using. I'd like >>>>>>>>>>>>>> to run this. >>>>>>>>>>>>>> >>>>>>>>>>>>>> does that cep problem is now all right that we were trying to >>>>>>>>>>>>>>> fix. I am still using those pre-build versions. If so i can >>>>>>>>>>>>>>> merge with the >>>>>>>>>>>>>>> latest one. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'll check this and let you know. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Can we arrange a meeting (preferably in WSO2 offices) in next >>>>>>>>>>>>>> week with ML team members as well. Coding period begins on next >>>>>>>>>>>>>> Monday, so >>>>>>>>>>>>>> it's better to get overall feedback from others and discuss more >>>>>>>>>>>>>> about the >>>>>>>>>>>>>> project. Let me know convenient time slots for you. I'll arrange >>>>>>>>>>>>>> a meeting >>>>>>>>>>>>>> with ML team. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:53 AM, Mahesh Dananjaya < >>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>> Ok. I will check it.you have sent me those relevant >>>>>>>>>>>>>>> references and i am working on that thing.thank you. does that >>>>>>>>>>>>>>> cep problem >>>>>>>>>>>>>>> is now all right that we were trying to fix. I am still using >>>>>>>>>>>>>>> those >>>>>>>>>>>>>>> pre-build versions. If so i can merge with the latest >>>>>>>>>>>>>>> one.thanks. >>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:44 AM, Maheshakya Wijewardena < >>>>>>>>>>>>>>> mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> You don't actually have to implement anything in spark >>>>>>>>>>>>>>>> streaming. Try to understand how streaming data is handled in >>>>>>>>>>>>>>>> and the >>>>>>>>>>>>>>>> specifics of the underlying algorithms in streaming. >>>>>>>>>>>>>>>> What we want to do is having the similar algorithms that >>>>>>>>>>>>>>>> support CEP event streams with siddhi. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Wed, May 18, 2016 at 9:38 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>> Did you check the repo. I will add recent works today.And >>>>>>>>>>>>>>>>> also i was going through the Java docs related to spark >>>>>>>>>>>>>>>>> streaming work. It >>>>>>>>>>>>>>>>> is with that scala API. thank you. >>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> On Tue, May 17, 2016 at 10:11 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>> I have gone through the Java Docs and run some of the >>>>>>>>>>>>>>>>>> Spark examples on spark shell which are paramount improtant >>>>>>>>>>>>>>>>>> for our work. >>>>>>>>>>>>>>>>>> Then i have been writing my codes to check the Linear >>>>>>>>>>>>>>>>>> regression, K means >>>>>>>>>>>>>>>>>> for streaming. please check my git repo [1]. I think now i >>>>>>>>>>>>>>>>>> have to ask on >>>>>>>>>>>>>>>>>> dev regarding the capturing event streams for our work. I >>>>>>>>>>>>>>>>>> will update the >>>>>>>>>>>>>>>>>> recent things on git. check the park-example directory for >>>>>>>>>>>>>>>>>> java. examples >>>>>>>>>>>>>>>>>> run on git shell is not included there. In my case i think i >>>>>>>>>>>>>>>>>> have to build >>>>>>>>>>>>>>>>>> mini batches from data streams that comes as individual >>>>>>>>>>>>>>>>>> samples. Now i am >>>>>>>>>>>>>>>>>> working on some coding to collect mini batches from data >>>>>>>>>>>>>>>>>> streams.thank you. >>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Tue, May 17, 2016 at 10:10 AM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>> I have gone through the Java Docs and run some of the >>>>>>>>>>>>>>>>>>> Spark examples on spark shell which are paramount improtant >>>>>>>>>>>>>>>>>>> for our work. >>>>>>>>>>>>>>>>>>> Then i have been writing my codes to check the Linear >>>>>>>>>>>>>>>>>>> regression, K means >>>>>>>>>>>>>>>>>>> for streaming. please check my git repo [1]. I think now i >>>>>>>>>>>>>>>>>>> have to ask on >>>>>>>>>>>>>>>>>>> dev regarding the capturing event streams for our work. I >>>>>>>>>>>>>>>>>>> will update the >>>>>>>>>>>>>>>>>>> recent things on git. check the park-example directory for >>>>>>>>>>>>>>>>>>> java. examples >>>>>>>>>>>>>>>>>>> run on git shell is not included there. In my case i think >>>>>>>>>>>>>>>>>>> i have to build >>>>>>>>>>>>>>>>>>> mini batches from data streams that comes as individual >>>>>>>>>>>>>>>>>>> samples. Now i am >>>>>>>>>>>>>>>>>>> working on some coding to collect mini batches from data >>>>>>>>>>>>>>>>>>> streams.thank you. >>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>> [1]https://github.com/dananjayamahesh/GSOC2016 >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Mon, May 16, 2016 at 1:19 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>> thank you. i will update the repo today.thank you.i >>>>>>>>>>>>>>>>>>>> changed the carbon ml siddhi extention and see how the >>>>>>>>>>>>>>>>>>>> changes are >>>>>>>>>>>>>>>>>>>> effecting. i will update the progress as soon as >>>>>>>>>>>>>>>>>>>> possible.thank you. i had >>>>>>>>>>>>>>>>>>>> some problem in spark mllib dependency. i was fixing that. >>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>> p.s: do i need to maintain a blog? >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Mon, May 16, 2016 at 10:02 AM, Maheshakya >>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Sorry for replying late. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thank you for the update. I believe you have done some >>>>>>>>>>>>>>>>>>>>> implementations with with Spark MLLIb algorithms in >>>>>>>>>>>>>>>>>>>>> streaming fashion as we >>>>>>>>>>>>>>>>>>>>> have discussed. If so, can you please share your code in >>>>>>>>>>>>>>>>>>>>> a Github repo. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Now i want to implements some machine learning >>>>>>>>>>>>>>>>>>>>>> algorithms with importing mllib and want to run within >>>>>>>>>>>>>>>>>>>>>> your code base >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> For the moment you can try out editing the same class >>>>>>>>>>>>>>>>>>>>> PredictStreamProcessor in the siddhi extension in >>>>>>>>>>>>>>>>>>>>> carbon-ml. Later we will >>>>>>>>>>>>>>>>>>>>> add this separately. You should be able to add >>>>>>>>>>>>>>>>>>>>> org.apache.spark.mllib. >>>>>>>>>>>>>>>>>>>>> classes to there. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> And i want to see how event streams are coming from >>>>>>>>>>>>>>>>>>>>>> cep. As i think it is not in a RDD format since it is >>>>>>>>>>>>>>>>>>>>>> arriving as the >>>>>>>>>>>>>>>>>>>>>> individual samples. I will send a email to dev asking >>>>>>>>>>>>>>>>>>>>>> about how to get the >>>>>>>>>>>>>>>>>>>>>> streams. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Please pay attention to length[1] and lengthbatch[1] >>>>>>>>>>>>>>>>>>>>> inbuilt windows in siddhi. What you need to write are >>>>>>>>>>>>>>>>>>>>> functions similar to >>>>>>>>>>>>>>>>>>>>> a custom aggregate function[2]. >>>>>>>>>>>>>>>>>>>>> When you send the email to dev list, explain your >>>>>>>>>>>>>>>>>>>>> requirement. You need to get a set of event with from a >>>>>>>>>>>>>>>>>>>>> stream with a >>>>>>>>>>>>>>>>>>>>> specified window size (number of events). Then build a >>>>>>>>>>>>>>>>>>>>> model within that >>>>>>>>>>>>>>>>>>>>> function. You also need to retain the data (learned >>>>>>>>>>>>>>>>>>>>> weights, cluster >>>>>>>>>>>>>>>>>>>>> centers, etc.) from the previous window to use in the >>>>>>>>>>>>>>>>>>>>> current window. Ask >>>>>>>>>>>>>>>>>>>>> what can be the most suitable option for this among the >>>>>>>>>>>>>>>>>>>>> set of siddhi >>>>>>>>>>>>>>>>>>>>> extensions given. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Inbuilt+Windows#InbuiltWindows-lengthlength >>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/CEP400/Writing+a+Custom+Aggregate+Function >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Wed, May 11, 2016 at 1:43 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> ---------- Forwarded message ---------- >>>>>>>>>>>>>>>>>>>>>> From: Mahesh Dananjaya <dananjayamah...@gmail.com> >>>>>>>>>>>>>>>>>>>>>> Date: Wed, May 11, 2016 at 1:43 PM >>>>>>>>>>>>>>>>>>>>>> Subject: Re: [Dev] GSOC2016: [ML][CEP] Predictive >>>>>>>>>>>>>>>>>>>>>> analytic with online data for WSO2 Machine Learner >>>>>>>>>>>>>>>>>>>>>> To: Maheshakya Wijewardena <mahesha...@wso2.com> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>> sorry for not updating. I did what you wanted me to >>>>>>>>>>>>>>>>>>>>>> do. I checked the code base and train functions. I went >>>>>>>>>>>>>>>>>>>>>> through those java >>>>>>>>>>>>>>>>>>>>>> docs. I went through the carbon-ml current >>>>>>>>>>>>>>>>>>>>>> implementation of LG and K-Mean. >>>>>>>>>>>>>>>>>>>>>> And i had Apache Spark and i tried with several >>>>>>>>>>>>>>>>>>>>>> examples. Now i want to >>>>>>>>>>>>>>>>>>>>>> implements some machine learning algorithms with >>>>>>>>>>>>>>>>>>>>>> importing mllib and want >>>>>>>>>>>>>>>>>>>>>> to run within your code base. Can you help me with that. >>>>>>>>>>>>>>>>>>>>>> And i want to see how event streams are coming from >>>>>>>>>>>>>>>>>>>>>> cep. As i think it is not in a RDD format since it is >>>>>>>>>>>>>>>>>>>>>> arriving as the >>>>>>>>>>>>>>>>>>>>>> individual samples. I will send a email to dev asking >>>>>>>>>>>>>>>>>>>>>> about how to get the >>>>>>>>>>>>>>>>>>>>>> streams. I debugged many of those functions in the code >>>>>>>>>>>>>>>>>>>>>> base. So need >>>>>>>>>>>>>>>>>>>>>> further instructions to proceed.thank you. >>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Wed, May 11, 2016 at 10:32 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Any update on your progress? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Wed, May 4, 2016 at 8:35 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> is that "Put break points in train methods in >>>>>>>>>>>>>>>>>>>>>>>>> Linear Regression class" means the spark/algorithms/ >>>>>>>>>>>>>>>>>>>>>>>>> LinearRegrassion.java >>>>>>>>>>>>>>>>>>>>>>>>> class in the org.wso2.carbon.ml.core? is that the >>>>>>>>>>>>>>>>>>>>>>>>> correct file? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Yes, this is the correct place. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> You can refer to spark programming guide[1][2] as >>>>>>>>>>>>>>>>>>>>>>>> well as our ML code base when you try those algorithms >>>>>>>>>>>>>>>>>>>>>>>> out. Please try to >>>>>>>>>>>>>>>>>>>>>>>> do rough implementations of the streaming versions of >>>>>>>>>>>>>>>>>>>>>>>> linear regression, >>>>>>>>>>>>>>>>>>>>>>>> logistic regression and k-means clustering as we have >>>>>>>>>>>>>>>>>>>>>>>> discussed in the >>>>>>>>>>>>>>>>>>>>>>>> proposal in plain Java. It's better if you can create >>>>>>>>>>>>>>>>>>>>>>>> a git repo and share >>>>>>>>>>>>>>>>>>>>>>>> your code once you have made some progress. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Were you able debug and understand the flow of the >>>>>>>>>>>>>>>>>>>>>>>> ML siddhi extension? I hope you haven't encountered >>>>>>>>>>>>>>>>>>>>>>>> more errors after >>>>>>>>>>>>>>>>>>>>>>>> switching the released version of CEP. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Is this Friday okay for you? Afternoon at 2:00 pm? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>> http://spark.apache.org/docs/latest/programming-guide.html >>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>> http://spark.apache.org/docs/latest/mllib-guide.html >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> On Wed, May 4, 2016 at 1:07 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>> I have been looking into some algorithms related >>>>>>>>>>>>>>>>>>>>>>>>> to stochastic gradient descent based >>>>>>>>>>>>>>>>>>>>>>>>> algorithms.anything i should focus >>>>>>>>>>>>>>>>>>>>>>>>> please let me know.Ans also i will be available for >>>>>>>>>>>>>>>>>>>>>>>>> calling this week and >>>>>>>>>>>>>>>>>>>>>>>>> next week.thank you. >>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 5:05 PM, Mahesh Dananjaya < >>>>>>>>>>>>>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>> thank you.that's good. i have been trying to fix >>>>>>>>>>>>>>>>>>>>>>>>>> that for couple of days. please inform me when it >>>>>>>>>>>>>>>>>>>>>>>>>> will be fixed.now i have >>>>>>>>>>>>>>>>>>>>>>>>>> been testing the ML algorithms and trying to >>>>>>>>>>>>>>>>>>>>>>>>>> identify the flow and the >>>>>>>>>>>>>>>>>>>>>>>>>> hierarchy. is that "Put break points in train >>>>>>>>>>>>>>>>>>>>>>>>>> methods in Linear Regression >>>>>>>>>>>>>>>>>>>>>>>>>> class" means the spark/algorithms/ >>>>>>>>>>>>>>>>>>>>>>>>>> LinearRegrassion.java class in the >>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.core? is that the correct file? >>>>>>>>>>>>>>>>>>>>>>>>>> And also i am planning to write some programs to >>>>>>>>>>>>>>>>>>>>>>>>>> use apache spark mllib algorithms. and i refer to >>>>>>>>>>>>>>>>>>>>>>>>>> [1] and some wso2 >>>>>>>>>>>>>>>>>>>>>>>>>> documentations to get some idea about ML >>>>>>>>>>>>>>>>>>>>>>>>>> structure.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> [1]nirmalfdo.blogspot.com >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 4:36 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> I have checked. It seems the issue you have >>>>>>>>>>>>>>>>>>>>>>>>>>> encountered is cause only in the current >>>>>>>>>>>>>>>>>>>>>>>>>>> development branch of the >>>>>>>>>>>>>>>>>>>>>>>>>>> product-cep. It doesn't identify the ML siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>> extension as an extension. >>>>>>>>>>>>>>>>>>>>>>>>>>> ML siddhi extension works fine in the latest >>>>>>>>>>>>>>>>>>>>>>>>>>> release of CEP (4.1.0) [1]. >>>>>>>>>>>>>>>>>>>>>>>>>>> Until we figure out the reason and come up with >>>>>>>>>>>>>>>>>>>>>>>>>>> a solution, can you use the latest CEP release for >>>>>>>>>>>>>>>>>>>>>>>>>>> your work. It's fine to >>>>>>>>>>>>>>>>>>>>>>>>>>> use that since you haven't started actual >>>>>>>>>>>>>>>>>>>>>>>>>>> development yet. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>> http://wso2.com/products/complex-event-processor/ >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 3:19 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is is vital to use those local repo in my >>>>>>>>>>>>>>>>>>>>>>>>>>>>> upcoming implementation? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes. The remote p2-repo contains the p2-repos >>>>>>>>>>>>>>>>>>>>>>>>>>>> of released versions. What you have to develop on >>>>>>>>>>>>>>>>>>>>>>>>>>>> is the current master of >>>>>>>>>>>>>>>>>>>>>>>>>>>> the carbon-ml and product-ml. You can try out with >>>>>>>>>>>>>>>>>>>>>>>>>>>> the modification I have >>>>>>>>>>>>>>>>>>>>>>>>>>>> suggested. In the meantime, I'll verify whether >>>>>>>>>>>>>>>>>>>>>>>>>>>> the current repos are >>>>>>>>>>>>>>>>>>>>>>>>>>>> working as expected. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am trying to debug the carbon-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.core by putting some break >>>>>>>>>>>>>>>>>>>>>>>>>>>>> point in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/Linear Regression >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> It's great that you have started looking at the >>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation of linear regression as well. Put >>>>>>>>>>>>>>>>>>>>>>>>>>>> break points in train >>>>>>>>>>>>>>>>>>>>>>>>>>>> methods in LinearRegression class. This is being >>>>>>>>>>>>>>>>>>>>>>>>>>>> used when you run linear >>>>>>>>>>>>>>>>>>>>>>>>>>>> regression from UI. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I can see some comments left behind for >>>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming algo as well.thank you >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> You may be referring to the linear regression >>>>>>>>>>>>>>>>>>>>>>>>>>>> with SGD model. Here, there's no retraining with >>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming data involved. >>>>>>>>>>>>>>>>>>>>>>>>>>>> The SGD with minibatches is used to train the >>>>>>>>>>>>>>>>>>>>>>>>>>>> model with the data set only >>>>>>>>>>>>>>>>>>>>>>>>>>>> once. >>>>>>>>>>>>>>>>>>>>>>>>>>>> What you have to do is create a similar >>>>>>>>>>>>>>>>>>>>>>>>>>>> mechanism to involve streaming data and retrain >>>>>>>>>>>>>>>>>>>>>>>>>>>> models. We will get to that >>>>>>>>>>>>>>>>>>>>>>>>>>>> part once you get comfortable with siddhi >>>>>>>>>>>>>>>>>>>>>>>>>>>> extensions. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> BTW, is it possible for you to join a call on >>>>>>>>>>>>>>>>>>>>>>>>>>>> this Friday or in the next week. We'll try to >>>>>>>>>>>>>>>>>>>>>>>>>>>> resolve your current issues >>>>>>>>>>>>>>>>>>>>>>>>>>>> and discuss further on project. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 1:03 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it ok to go with p2 repo at >>>>>>>>>>>>>>>>>>>>>>>>>>>>> http://product-dist.wso2.com/p2/carbon/releases/wilkes/features/ >>>>>>>>>>>>>>>>>>>>>>>>>>>>> rather than the P2-repo at >>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml/modules/p2-profile/target/p2-repo in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> local repo.What is the impact?.Is is vital to use >>>>>>>>>>>>>>>>>>>>>>>>>>>>> those local repo in my >>>>>>>>>>>>>>>>>>>>>>>>>>>>> upcoming implementation?so i was trying to give >>>>>>>>>>>>>>>>>>>>>>>>>>>>> remote p2 repo to cep built >>>>>>>>>>>>>>>>>>>>>>>>>>>>> by source and trying to debug the cep ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>> extension and got the same error >>>>>>>>>>>>>>>>>>>>>>>>>>>>> as yesterday. But pre-built product is working >>>>>>>>>>>>>>>>>>>>>>>>>>>>> fine. Therefore now i am >>>>>>>>>>>>>>>>>>>>>>>>>>>>> trying as you described in the last email. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> And also i am trying to debug the carbon-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.core by putting some break >>>>>>>>>>>>>>>>>>>>>>>>>>>>> point in the >>>>>>>>>>>>>>>>>>>>>>>>>>>>> spark/algorithms/Linear Regression. I am trying >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to trigger it with >>>>>>>>>>>>>>>>>>>>>>>>>>>>> product-ml project with data set. Does that >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linear Regression also in UI is >>>>>>>>>>>>>>>>>>>>>>>>>>>>> consuming that spark algorithms or is it in >>>>>>>>>>>>>>>>>>>>>>>>>>>>> another place? I can see some >>>>>>>>>>>>>>>>>>>>>>>>>>>>> comments left behind for streaming algo as >>>>>>>>>>>>>>>>>>>>>>>>>>>>> well.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, May 3, 2016 at 9:35 AM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The earlier error you have mentioned may >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> occur due to the incompatible Siddhi versions in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ML p2-repo and CEP, when >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you add the p2-repo from prodcut-ml you built. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Current siddhi version in product-cep is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3.0.6-SNAPSHOT[1], but in ML it's 3.0.2. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Can you try changing the siddhi.version in >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml/pom.xml to 3.0.6-SNAPSHOT, build >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml, then build product-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again. After this, add p2-repo as a local >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repository again to fresh CEP >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pack and try it out. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, May 2, 2016 at 7:02 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Maheshakya, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now i remotely debug the CEP extension for >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ML Prediction. What i did was, i have all the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pre-build version of CEP and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ML. Therefore i used that built CEP and did the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> same thing that i was doing >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with the source code. I think the only change i >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> did was install those >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> packages from remote p2 repo. This was work >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> fine and i debugged the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml org.wso2.carbon.ml.siddhi.extension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> as described in the [1]. So >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now i have to try same thing with the build by >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> source content.thank you. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> BR, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 5:49 PM, Maheshakya >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Wijewardena <mahesha...@wso2.com> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi Mahesh, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Congratulations and welcome to GSoC 2016. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> You did a great job in preparing the proposal. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Now it's time to dig deep >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and get started with the project. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> First of all you need to familiarize with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the code base. We have agreed to implement >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this with CEP event streams. We >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already have a CEP extension for predictions >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1][2]. Go through this >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> implementation and familiarize your self with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that. You need to understand >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> how: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. Even streams are consumed >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. predictions are made from individual >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> event >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 3. Results are sent back >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Get WSO2 ML and CEP sources (You may use >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> latest released version of CEP) and build the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> products. Get both >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> carbon-ml[3] and product-ml[4] masters and >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> create new branches for your >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work from masters. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> After you build the products, you may need >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to do remote debugging[5] to understand the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> flow. So please follow an >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> example of real time prediction with ML with >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> debugging and get some idea. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The component you need to debug is >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.wso2.carbon.ml.siddhi.extension. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Next tasks would be implementing online >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> learning algorithms in plain java with spark >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ml lib and integrating those >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to ML. We also need to come up with a proper >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and detailed architecture to >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> employ those algorithms in ML. Getting >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> familiar with the aforementioned >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sections would give you some insight on how >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this should be implemented. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> So please try to get a quick grasp then you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can start the implementation. Let us know if >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> you have any questions or you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get stuck somewhere. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Also, please always add WSO2 developer's >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> list as well when you communicate with us >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regarding the project so that you >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can get opinions and feedback from others as >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> well. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Best regards. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/extensions/org.wso2.carbon.ml.siddhi.extension >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [3] https://github.com/wso2/carbon-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [4] https://github.com/wso2/product-ml >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> [5] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://dzone.com/articles/how-debug-wso2-carbon-kernel >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 3:33 PM, Mahesh >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Dananjaya <dananjayamah...@gmail.com> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> thank you for accepting my GSOC 2016 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> proposal and i am looking forward for the >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> further instruction and project >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> continuation. thank you very much. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> regards, >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Mahesh. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>>>>>>>>>> Dev@wso2.org >>>>>>>>>>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>>>>>>> mahesha...@wso2.com >>>>>>>>>>>>>> +94711228855 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>>> mahesha...@wso2.com >>>>>>>>> +94711228855 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>> mahesha...@wso2.com >>>>>>>> +94711228855 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Pruthuvi Maheshakya Wijewardena >>>> mahesha...@wso2.com >>>> +94711228855 >>>> >>>> >>>> >>> >> >> >> -- >> *Supun Sethunga* >> Senior Software Engineer >> WSO2, Inc. >> http://wso2.com/ >> lean | enterprise | middleware >> Mobile : +94 716546324 >> > >
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
