---------- Forwarded message ----------
From: Mahesh Dananjaya <dananjayamah...@gmail.com>
Date: Mon, May 2, 2016 at 5:30 PM
Subject: Re: [Dev] GSOC2016: [ML][CEP] Predictive analytic with online data
for WSO2 Machine Learner
To: Maheshakya Wijewardena <mahesha...@wso2.com>
Hi maheshakya,
may i try with remote p2 repo rather than my local one?can you send me the
url for relevant one. When i was building cep by source there was a error
in testing phase. it was built successfully. But for the time i forget that
one and use that built version after.
BR,
Mahesh.
On Mon, May 2, 2016 at 5:27 PM, Mahesh Dananjaya <dananjayamah...@gmail.com>
wrote:
> Hi Maheshakya,
> I did several changes. i generated the ml model and then start the cep.and
> also i changed the expression to relevant location of the model as,
>
> from
> InputStream#ml:predict('registry:/home/mahesh/GSOC/WSO2/product-ml/modules/distribution/target/wso2ml-2.0.0-SNAPSHOT/models/PimaIndian.Model.2016-05-02_16-40-12','double')
>
> But still when i click the "Validate Query Expression" the error is
> popping up."No extension exist for StreamFunctionExtension{namespace='ml'}
> in execution plan "ExecutionPlan"".
>
> the only source is [1]that mentioned about this kind of error as i
> mentioned. is that expression is correct now?
> BR,
> mahesh.
>
> [1]https://wso2.org/jira/browse/CEP-1400
>
> On Mon, May 2, 2016 at 3:49 PM, Maheshakya Wijewardena <
> mahesha...@wso2.com> wrote:
>
>> You can first train a model then shut down the ML server. For prediction
>> with CEP, you need only the file persisted file in the {ML_HOME}/models/.
>>
>> But in your actual implementation, you will have to have a running ML
>> server as well because training should be done continuously.
>>
>> On Mon, May 2, 2016 at 3:33 PM, Mahesh Dananjaya <
>> dananjayamah...@gmail.com> wrote:
>>
>>> Hi maheshakya,
>>> are both ML should be running simultaneously with cep for this to
>>> happen.an prior to cep execution we should generate the model? then i
>>> think i made a mistake there.i shut down the ml server and start the cep.So
>>> is that wy tis error is there.
>>> BR,
>>> Mahesh.
>>>
>>> On Mon, May 2, 2016 at 3:27 PM, Mahesh Dananjaya <
>>> dananjayamah...@gmail.com> wrote:
>>>
>>>>
>>>> Hi maheshakya,
>>>> I have installed them correctly.now I am trying to debug the siddhi
>>>> extention with the cep as the [1] describes. But when i created an input
>>>> stream and a predictionStream (output stream). when i was trying to create
>>>> new execution plan with above streams i got error when i clicked "Validate
>>>> Query Expression".Error was,
>>>> Error:
>>>> No extension exist for StreamFunctionExtension{namespace='ml'} in
>>>> execution plan "ExecutionPlan"
>>>>
>>>> and my expression is like a
>>>>
>>>> /* Enter a unique ExecutionPlan */
>>>> @Plan:name('ExecutionPlan')
>>>>
>>>> /* Enter a unique description for ExecutionPlan */
>>>> -- @Plan:description('ExecutionPlan')
>>>>
>>>> /* define streams/tables and write queries here ... */
>>>>
>>>> @Import('InputStream:1.0.0')
>>>> define stream InputStream (NumPregnancies double, TSFT double, DPF
>>>> double, BMI double, DBP double, PG2 double, Age double, SI2 double);
>>>>
>>>> @Export('PredictionStream:1.0.0')
>>>> define stream PredictionSTream (NumPregnancies double, TSFT double, DPF
>>>> double, BMI double, DBP double, PG2 double, Age double, SI2 double, Class
>>>> double);
>>>>
>>>> from
>>>> InputStream#ml:predict('file:///home/mahesh/GSOC/WSO2/data-set/pima-indian-diabetes.data','double')
>>>> select *
>>>> insert into PredictionStream
>>>>
>>>>
>>>> i used file instead of registry. And i referred to the [2] and there
>>>> they mention that solution for fixing CEP is running on distributed mode
>>>> with apache Storm cluster.
>>>>
>>>> 1. Is that CEP i built is originally run as distributed mode?
>>>> 2. Is this cuased by an not having sudo privilleges in current user
>>>> when installing ML features onto CEP?
>>>> 3.Is this the correct way to give file to CEP.
>>>>
>>>> [1]
>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension
>>>>
>>>> [2]https://wso2.org/jira/browse/CEP-1400
>>>>
>>>> BR,
>>>> Mahesh.
>>>>
>>>>
>>>> On Mon, May 2, 2016 at 12:35 PM, Maheshakya Wijewardena <
>>>> mahesha...@wso2.com> wrote:
>>>>
>>>>> Hi Mahesh,
>>>>>
>>>>> If you have built product-ml, you can find the P2-repo at
>>>>> product-ml/modules/p2-profile/target/p2-repo
>>>>> Add this folder as a local repository.
>>>>> After that, you should be able to see the ML features.
>>>>>
>>>>> Best regards.
>>>>>
>>>>> On Mon, May 2, 2016 at 12:24 PM, Mahesh Dananjaya <
>>>>> dananjayamah...@gmail.com> wrote:
>>>>>
>>>>>> Hi Maheshakya,
>>>>>> Since i already have carbon-ml built in my pc can i use my local
>>>>>> repository to install those features in to CEP.is that correct.thank you.
>>>>>> regards,
>>>>>> Mahesh.
>>>>>>
>>>>>> On Mon, May 2, 2016 at 12:20 PM, Mahesh Dananjaya <
>>>>>> dananjayamah...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Maheshakya,
>>>>>>> Can you please tell me how to find the most recent p2 repository URL
>>>>>>> to add machine learner Core, Machine learner commons, Machine learner
>>>>>>> database service and ML Siddhi extension to add as features in CEP as
>>>>>>> describes in the [1]. When i use
>>>>>>> http://product-dist.wso2.com/p2/carbon/releases/4.2.0/ URL those
>>>>>>> features are not visible in the CEP.Is that not he most recent one.
>>>>>>> BR,
>>>>>>> Mahesh.
>>>>>>>
>>>>>>> [1]
>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension
>>>>>>>
>>>>>>> On Mon, May 2, 2016 at 11:28 AM, Mahesh Dananjaya <
>>>>>>> dananjayamah...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Maheshakya,
>>>>>>>> sorry for the incomplete message.I have set up the dev environment
>>>>>>>> and now i am trying to remotely debug. The following steps were done.
>>>>>>>> 1. build product-cep, carbon-ml and product-ml by source.
>>>>>>>> 2. go through their code bases and trying to understand the way and
>>>>>>>> the flow you developed.
>>>>>>>> 3. i have set up break point in org.wso2.carbon.ml.siddhi.extension
>>>>>>>> in carbon-ml
>>>>>>>> 4. start the ./wso2server.sh debug 5005 in the SNAPSHOT directory
>>>>>>>> of product-ml
>>>>>>>> 5. trying to trigger the break points with the [1] reference.break
>>>>>>>> points are placed in the PredictStreamProcessor.java file within the
>>>>>>>> extention.
>>>>>>>>
>>>>>>>> This is the way i followed. I was trying to remotely debug the ML
>>>>>>>> core by putting break-points in ml core.(org.wso2.carbon.ml.core) in
>>>>>>>> spark
>>>>>>>> java files. Is this the right way to do those things.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension
>>>>>>>>
>>>>>>>> On Mon, May 2, 2016 at 11:19 AM, Mahesh Dananjaya <
>>>>>>>> dananjayamah...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi maheshakya,
>>>>>>>>> I have set up the dev environment and now i am trying to remotely
>>>>>>>>> debug. The following steps were done.
>>>>>>>>> 1. build product-cep, carbon-ml and product-ml by source.
>>>>>>>>> 2. go through their code bases and trying to understand the way
>>>>>>>>> and the flow you developed.
>>>>>>>>> 3. i have set up break point in
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Apr 28, 2016 at 7:05 PM, Mahesh Dananjaya <
>>>>>>>>> dananjayamah...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Maheshakya,
>>>>>>>>>> ok.i got it.thank you.
>>>>>>>>>> regards,
>>>>>>>>>> Mahesh.
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 28, 2016 at 6:56 PM, Maheshakya Wijewardena <
>>>>>>>>>> mahesha...@wso2.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Mahesh,
>>>>>>>>>>>
>>>>>>>>>>> The links was an example of remote debugging WSO2 server. What
>>>>>>>>>>> you need to debug is org.wso2.carbon.ml.siddhi.extension in
>>>>>>>>>>> carbon-ml.
>>>>>>>>>>>
>>>>>>>>>>> Best regards.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Apr 28, 2016 at 4:52 PM, Mahesh Dananjaya <
>>>>>>>>>>> dananjayamah...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Maheshakya,
>>>>>>>>>>>> thank you for your help.i have already built all three sources
>>>>>>>>>>>> and now i am trying to get familiar with your code base. i even
>>>>>>>>>>>> build the
>>>>>>>>>>>> carbon-kernel by source.
>>>>>>>>>>>> As you mentioned [1] is related to debug the kernel, do i
>>>>>>>>>>>> really need to debug the carbon kernel in my case. I am trying to
>>>>>>>>>>>> remotely
>>>>>>>>>>>> debug ml and as i got it correct it is the same way as
>>>>>>>>>>>> reference[1, but not
>>>>>>>>>>>> the kernel.I can go with others.
>>>>>>>>>>>> BR,
>>>>>>>>>>>> mahesh.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://dzone.com/articles/how-debug-wso2-carbon-kernel
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 25, 2016 at 5:49 PM, Maheshakya Wijewardena <
>>>>>>>>>>>> mahesha...@wso2.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Mahesh,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Congratulations and welcome to GSoC 2016. You did a great job
>>>>>>>>>>>>> in preparing the proposal. Now it's time to dig deep and get
>>>>>>>>>>>>> started with
>>>>>>>>>>>>> the project.
>>>>>>>>>>>>>
>>>>>>>>>>>>> First of all you need to familiarize with the code base. We
>>>>>>>>>>>>> have agreed to implement this with CEP event streams. We already
>>>>>>>>>>>>> have a CEP
>>>>>>>>>>>>> extension for predictions [1][2]. Go through this implementation
>>>>>>>>>>>>> and
>>>>>>>>>>>>> familiarize your self with that. You need to understand how:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Even streams are consumed
>>>>>>>>>>>>> 2. predictions are made from individual event
>>>>>>>>>>>>> 3. Results are sent back
>>>>>>>>>>>>>
>>>>>>>>>>>>> Get WSO2 ML and CEP sources (You may use latest released
>>>>>>>>>>>>> version of CEP) and build the products. Get both carbon-ml[3] and
>>>>>>>>>>>>> product-ml[4] masters and create new branches for your work from
>>>>>>>>>>>>> masters.
>>>>>>>>>>>>>
>>>>>>>>>>>>> After you build the products, you may need to do remote
>>>>>>>>>>>>> debugging[5] to understand the flow. So please follow an example
>>>>>>>>>>>>> of real
>>>>>>>>>>>>> time prediction with ML with debugging and get some idea. The
>>>>>>>>>>>>> component you
>>>>>>>>>>>>> need to debug is org.wso2.carbon.ml.siddhi.extension.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Next tasks would be implementing online learning algorithms in
>>>>>>>>>>>>> plain java with spark ml lib and integrating those to ML. We also
>>>>>>>>>>>>> need to
>>>>>>>>>>>>> come up with a proper and detailed architecture to employ those
>>>>>>>>>>>>> algorithms
>>>>>>>>>>>>> in ML. Getting familiar with the aforementioned sections would
>>>>>>>>>>>>> give you
>>>>>>>>>>>>> some insight on how this should be implemented.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So please try to get a quick grasp then you can start the
>>>>>>>>>>>>> implementation. Let us know if you have any questions or you get
>>>>>>>>>>>>> stuck
>>>>>>>>>>>>> somewhere.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, please always add WSO2 developer's list as well when you
>>>>>>>>>>>>> communicate with us regarding the project so that you can get
>>>>>>>>>>>>> opinions and
>>>>>>>>>>>>> feedback from others as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best regards.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1]
>>>>>>>>>>>>> https://docs.wso2.com/display/ML110/WSO2+CEP+Extension+for+ML+Predictions#WSO2CEPExtensionforMLPredictions-Siddhisyntaxfortheextension
>>>>>>>>>>>>>
>>>>>>>>>>>>> [2]
>>>>>>>>>>>>> https://github.com/wso2/carbon-ml/tree/master/components/extensions/org.wso2.carbon.ml.siddhi.extension
>>>>>>>>>>>>>
>>>>>>>>>>>>> [3] https://github.com/wso2/carbon-ml
>>>>>>>>>>>>>
>>>>>>>>>>>>> [4] https://github.com/wso2/product-ml
>>>>>>>>>>>>>
>>>>>>>>>>>>> [5] https://dzone.com/articles/how-debug-wso2-carbon-kernel
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Apr 25, 2016 at 3:33 PM, Mahesh Dananjaya <
>>>>>>>>>>>>> dananjayamah...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>> thank you for accepting my GSOC 2016 proposal and i am
>>>>>>>>>>>>>> looking forward for the further instruction and project
>>>>>>>>>>>>>> continuation. thank
>>>>>>>>>>>>>> you very much.
>>>>>>>>>>>>>> regards,
>>>>>>>>>>>>>> Mahesh.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena
>>>>>>>>>>>>> mahesha...@wso2.com
>>>>>>>>>>>>> +94711228855
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Pruthuvi Maheshakya Wijewardena
>>>>>>>>>>> mahesha...@wso2.com
>>>>>>>>>>> +94711228855
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Pruthuvi Maheshakya Wijewardena
>>>>> mahesha...@wso2.com
>>>>> +94711228855
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Pruthuvi Maheshakya Wijewardena
>> mahesha...@wso2.com
>> +94711228855
>>
>>
>>
>
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
