Hi Mahesh, is that "Put break points in train methods in Linear Regression class" > means the spark/algorithms/ LinearRegrassion.java class in the > org.wso2.carbon.ml.core? is that the correct file?
Yes, this is the correct place. You can refer to spark programming guide[1][2] as well as our ML code base when you try those algorithms out. Please try to do rough implementations of the streaming versions of linear regression, logistic regression and k-means clustering as we have discussed in the proposal in plain Java. It's better if you can create a git repo and share your code once you have made some progress. Were you able debug and understand the flow of the ML siddhi extension? I hope you haven't encountered more errors after switching the released version of CEP. Is this Friday okay for you? Afternoon at 2:00 pm? Best regards. Best regards. [1] http://spark.apache.org/docs/latest/programming-guide.html [2] http://spark.apache.org/docs/latest/mllib-guide.html On Wed, May 4, 2016 at 1:07 PM, Mahesh Dananjaya <dananjayamah...@gmail.com> wrote: > Hi Maheshakya, > I have been looking into some algorithms related to stochastic gradient > descent based algorithms.anything i should focus please let me know.Ans > also i will be available for calling this week and next week.thank you. > BR, > Mahesh. > > On Tue, May 3, 2016 at 5:05 PM, Mahesh Dananjaya < > dananjayamah...@gmail.com> wrote: > >> Hi Maheshakya, >> thank you.that's good. i have been trying to fix that for couple of days. >> please inform me when it will be fixed.now i have been testing the ML >> algorithms and trying to identify the flow and the hierarchy. is that "Put >> break points in train methods in Linear Regression class" means the >> spark/algorithms/ LinearRegrassion.java class in the >> org.wso2.carbon.ml.core? is that the correct file? >> And also i am planning to write some programs to use apache spark mllib >> algorithms. and i refer to [1] and some wso2 documentations to get some >> idea about ML structure.thank you. >> >> BR, >> Mahesh. >> >> [1]nirmalfdo.blogspot.com >> >> On Tue, May 3, 2016 at 4:36 PM, Maheshakya Wijewardena < >> mahesha...@wso2.com> wrote: >> >>> Hi Mahesh, >>> >>> I have checked. It seems the issue you have encountered is cause only in >>> the current development branch of the product-cep. It doesn't identify the >>> ML siddhi extension as an extension. ML siddhi extension works fine in the >>> latest release of CEP (4.1.0) [1]. >>> Until we figure out the reason and come up with a solution, can you use >>> the latest CEP release for your work. It's fine to use that since you >>> haven't started actual development yet. >>> >>> Best regards. >>> >>> [1] http://wso2.com/products/complex-event-processor/ >>> >>> On Tue, May 3, 2016 at 3:19 PM, Maheshakya Wijewardena < >>> mahesha...@wso2.com> wrote: >>> >>>> Hi Mahesh, >>>> >>>> >>>>> Is is vital to use those local repo in my upcoming implementation? >>>> >>>> >>>> Yes. The remote p2-repo contains the p2-repos of released versions. >>>> What you have to develop on is the current master of the carbon-ml and >>>> product-ml. You can try out with the modification I have suggested. In the >>>> meantime, I'll verify whether the current repos are working as expected. >>>> >>>> And also i am trying to debug the carbon-ml org.wso2.carbon.ml.core by >>>>> putting some break point in the spark/algorithms/Linear Regression >>>> >>>> >>>> It's great that you have started looking at the implementation of >>>> linear regression as well. Put break points in train methods in >>>> LinearRegression class. This is being used when you run linear regression >>>> from UI. >>>> >>>> I can see some comments left behind for streaming algo as well.thank you >>>> >>>> >>>> You may be referring to the linear regression with SGD model. Here, >>>> there's no retraining with streaming data involved. The SGD with >>>> minibatches is used to train the model with the data set only once. >>>> What you have to do is create a similar mechanism to involve streaming >>>> data and retrain models. We will get to that part once you get comfortable >>>> with siddhi extensions. >>>> >>>> BTW, is it possible for you to join a call on this Friday or in the >>>> next week. We'll try to resolve your current issues and discuss further on >>>> project. >>>> >>>> Best regards. >>>> >>>> On Tue, May 3, 2016 at 1:03 PM, Mahesh Dananjaya < >>>> dananjayamah...@gmail.com> wrote: >>>> >>>>> Hi maheshakya, >>>>> Is it ok to go with p2 repo at >>>>> http://product-dist.wso2.com/p2/carbon/releases/wilkes/features/ >>>>> rather than the P2-repo at product-ml/modules/p2-profile/target/p2-repo in >>>>> local repo.What is the impact?.Is is vital to use those local repo in my >>>>> upcoming implementation?so i was trying to give remote p2 repo to cep >>>>> built >>>>> by source and trying to debug the cep ml extension and got the same error >>>>> as yesterday. But pre-built product is working fine. Therefore now i am >>>>> trying as you described in the last email. >>>>> >>>>> And also i am trying to debug the carbon-ml org.wso2.carbon.ml.core by >>>>> putting some break point in the spark/algorithms/Linear Regression. I am >>>>> trying to trigger it with product-ml project with data set. Does that >>>>> Linear Regression also in UI is consuming that spark algorithms or is it >>>>> in >>>>> another place? I can see some comments left behind for streaming algo as >>>>> well.thank you. >>>>> BR, >>>>> Mahesh. >>>>> >>>>> On Tue, May 3, 2016 at 9:35 AM, Maheshakya Wijewardena < >>>>> mahesha...@wso2.com> wrote: >>>>> >>>>>> Hi Mahesh, >>>>>> >>>>>> The earlier error you have mentioned may occur due to the >>>>>> incompatible Siddhi versions in ML p2-repo and CEP, when you add the >>>>>> p2-repo from prodcut-ml you built. >>>>>> Current siddhi version in product-cep is 3.0.6-SNAPSHOT[1], but in ML >>>>>> it's 3.0.2. >>>>>> >>>>>> Can you try changing the siddhi.version in carbon-ml/pom.xml to >>>>>> 3.0.6-SNAPSHOT, build carbon-ml, then build product-ml again. After this, >>>>>> add p2-repo as a local repository again to fresh CEP pack and try it out. >>>>>> >>>>>> Best regards. >>>>>> >>>>>> On Mon, May 2, 2016 at 7:02 PM, Mahesh Dananjaya < >>>>>> dananjayamah...@gmail.com> wrote: >>>>>> >>>>>>> Hi Maheshakya, >>>>>>> now i remotely debug the CEP extension for ML Prediction. What i did >>>>>>> was, i have all the pre-build version of CEP and ML. Therefore i used >>>>>>> that >>>>>>> built CEP and did the same thing that i was doing with the source code. >>>>>>> I >>>>>>> think the only change i did was install those packages from remote p2 >>>>>>> repo. >>>>>>> This was work fine and i debugged the carbon-ml >>>>>>> org.wso2.carbon.ml.siddhi.extension as described in the [1]. So now i >>>>>>> have >>>>>>> to try same thing with the build by source content.thank you. >>>>>>> >>>>>>> [1] >>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>> >>>>>>> BR, >>>>>>> Mahesh >>>>>>> >>>>>>> On Mon, Apr 25, 2016 at 5:49 PM, Maheshakya Wijewardena < >>>>>>> mahesha...@wso2.com> wrote: >>>>>>> >>>>>>>> Hi Mahesh, >>>>>>>> >>>>>>>> Congratulations and welcome to GSoC 2016. You did a great job in >>>>>>>> preparing the proposal. Now it's time to dig deep and get started with >>>>>>>> the >>>>>>>> project. >>>>>>>> >>>>>>>> First of all you need to familiarize with the code base. We have >>>>>>>> agreed to implement this with CEP event streams. We already have a CEP >>>>>>>> extension for predictions [1][2]. Go through this implementation and >>>>>>>> familiarize your self with that. You need to understand how: >>>>>>>> >>>>>>>> 1. Even streams are consumed >>>>>>>> 2. predictions are made from individual event >>>>>>>> 3. Results are sent back >>>>>>>> >>>>>>>> Get WSO2 ML and CEP sources (You may use latest released version of >>>>>>>> CEP) and build the products. Get both carbon-ml[3] and product-ml[4] >>>>>>>> masters and create new branches for your work from masters. >>>>>>>> >>>>>>>> After you build the products, you may need to do remote >>>>>>>> debugging[5] to understand the flow. So please follow an example of >>>>>>>> real >>>>>>>> time prediction with ML with debugging and get some idea. The >>>>>>>> component you >>>>>>>> need to debug is org.wso2.carbon.ml.siddhi.extension. >>>>>>>> >>>>>>>> Next tasks would be implementing online learning algorithms in >>>>>>>> plain java with spark ml lib and integrating those to ML. We also need >>>>>>>> to >>>>>>>> come up with a proper and detailed architecture to employ those >>>>>>>> algorithms >>>>>>>> in ML. Getting familiar with the aforementioned sections would give you >>>>>>>> some insight on how this should be implemented. >>>>>>>> >>>>>>>> So please try to get a quick grasp then you can start the >>>>>>>> implementation. Let us know if you have any questions or you get stuck >>>>>>>> somewhere. >>>>>>>> >>>>>>>> Also, please always add WSO2 developer's list as well when you >>>>>>>> communicate with us regarding the project so that you can get opinions >>>>>>>> and >>>>>>>> feedback from others as well. >>>>>>>> >>>>>>>> Best regards. >>>>>>>> >>>>>>>> [1] >>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension >>>>>>>> >>>>>>>> [2] >>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/extensions/org.wso2.carbon.ml.siddhi.extension >>>>>>>> >>>>>>>> [3] https://github.com/wso2/carbon-ml >>>>>>>> >>>>>>>> [4] https://github.com/wso2/product-ml >>>>>>>> >>>>>>>> [5] https://dzone.com/articles/how-debug-wso2-carbon-kernel >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Apr 25, 2016 at 3:33 PM, Mahesh Dananjaya < >>>>>>>> dananjayamah...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> thank you for accepting my GSOC 2016 proposal and i am looking >>>>>>>>> forward for the further instruction and project continuation. thank >>>>>>>>> you >>>>>>>>> very much. >>>>>>>>> regards, >>>>>>>>> Mahesh. >>>>>>>>> >>>>>>>>> -- >>>>>>>> Pruthuvi Maheshakya Wijewardena >>>>>>>> mahesha...@wso2.com >>>>>>>> +94711228855 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Pruthuvi Maheshakya Wijewardena >>>>>> mahesha...@wso2.com >>>>>> +94711228855 >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Pruthuvi Maheshakya Wijewardena >>>> mahesha...@wso2.com >>>> +94711228855 >>>> >>>> >>>> >>> >>> >>> -- >>> Pruthuvi Maheshakya Wijewardena >>> mahesha...@wso2.com >>> +94711228855 >>> >>> >>> >> > -- Pruthuvi Maheshakya Wijewardena mahesha...@wso2.com +94711228855
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
