Thanks Danula. ML team like to do the integration of this, since there're few things we need to clear up in ML side.
Can you please come up with a clearly explained document on the project work carried out during the summer? On Wed, Sep 2, 2015 at 12:41 AM, Danula Eranjith <hmdanu...@gmail.com> wrote: > Hi, > > I have made the changes you suggested at [1] and created a API to access > the wrangler service at [2]. > > Also added a new step in the wizard at [3] with the wrangler interface. > Please have a look. > > How can we create a sample of the dataset to be passed to wrangler? I > couldn't find any current implementation capable of this. > > Another concern is if we are adding the feature selection step after the > cleaning step, we need to reflect the changes done in step one at step two. > But since we do not apply transformations to RDD initially, we need to come > up an alternative approach. > > [1] - > https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler/src/main/java/org/wso2/carbon/ml/wrangler > > [2] - > https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java > <https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java> > > [3] - > https://github.com/danula/carbon-ml/blob/master/apps/ml/site/clean/clean.jag > > Thanks, > Danula > > > On Thu, Aug 27, 2015 at 9:41 AM, Danula Eranjith <hmdanu...@gmail.com> > wrote: > >> Basically script exported from Wrangler tool has list of operations. >> Wrangler class parse that script and create WranglerOperation object for >> each operation with its parameters. >> Then when WranglerOperation.executeOperation() is invoked, it creates the >> respective SparkOperation object and then applies operations to the JavaRDD >> >> On Thu, Aug 27, 2015 at 9:35 AM, Nirmal Fernando <nir...@wso2.com> wrote: >> >>> What does WranglerOperation class do? >>> >>> On Thu, Aug 27, 2015 at 9:24 AM, Danula Eranjith <hmdanu...@gmail.com> >>> wrote: >>> >>>> Currently Wrangler Operation is the class that holds details related to >>>> wrangler and SparkOperation contains the relevant Spark transformation. >>>> >>>> If we are changing SparkOperation as WranglerOperation, we need to >>>> rename the current WranglerOperation into something else. >>>> >>>> On Thu, Aug 27, 2015 at 9:18 AM, Nirmal Fernando <nir...@wso2.com> >>>> wrote: >>>> >>>>> /s/SparkOpration/SparkOperation >>>>> >>>>> May be as Supun said, I too think we should call them as >>>>> 'WranglerOperation'. >>>>> >>>>> On Thu, Aug 27, 2015 at 7:02 AM, Nirmal Fernando <nir...@wso2.com> >>>>> wrote: >>>>> >>>>>> Also, avoid static methods in transformations. >>>>>> >>>>>> On Thu, Aug 27, 2015 at 2:48 AM, Supun Sethunga <sup...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Danula, >>>>>>> >>>>>>> Few comments: >>>>>>> >>>>>>> - You might have to register the component in the OSGI >>>>>>> environment, to be able to call the services from a another >>>>>>> component. >>>>>>> Refer [1] on how to do this. >>>>>>> - Better to introduce an interface for WranglerOperation class. >>>>>>> - Add class level/ method level comments. >>>>>>> - Use logger in-place of System.out.println >>>>>>> >>>>>>> [1] >>>>>>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/ds/MLDatabaseServiceDS.java >>>>>>> >>>>>>> Thanks, >>>>>>> Supun >>>>>>> >>>>>>> On Wed, Aug 26, 2015 at 1:32 PM, Danula Eranjith < >>>>>>> hmdanu...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I added the component at [1] >>>>>>>> <https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler> >>>>>>>> Please have a look. >>>>>>>> >>>>>>>> [1] - >>>>>>>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler >>>>>>>> >>>>>>>> Danula >>>>>>>> >>>>>>>> On Tue, Aug 25, 2015 at 8:35 PM, Danula Eranjith < >>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Thanks Supun >>>>>>>>> >>>>>>>>> On Tue, Aug 25, 2015 at 7:25 PM, Supun Sethunga <sup...@wso2.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> You can integrate it to [1], by adding a new component >>>>>>>>>> "org.wso2.carbon.ml.wrangler". Each component is a carbon component. >>>>>>>>>> >>>>>>>>>> Please follow the naming conventions used in the other >>>>>>>>>> components, for package names and etc.. >>>>>>>>>> >>>>>>>>>> [1] https://github.com/wso2/carbon-ml/tree/master/components/ml >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Supun >>>>>>>>>> >>>>>>>>>> On Tue, Aug 25, 2015 at 7:33 AM, Danula Eranjith < >>>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> Can you suggest where I should be ideally integrating these files >>>>>>>>>>> [1] >>>>>>>>>>> <https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src> >>>>>>>>>>> in ML. >>>>>>>>>>> >>>>>>>>>>> [1] - >>>>>>>>>>> https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Danula >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> *Supun Sethunga* >>>>>>>>>> Software Engineer >>>>>>>>>> WSO2, Inc. >>>>>>>>>> http://wso2.com/ >>>>>>>>>> lean | enterprise | middleware >>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Supun Sethunga* >>>>>>> Software Engineer >>>>>>> WSO2, Inc. >>>>>>> http://wso2.com/ >>>>>>> lean | enterprise | middleware >>>>>>> Mobile : +94 716546324 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Team Lead - WSO2 Machine Learner >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Team Lead - WSO2 Machine Learner >>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>> >>> >>> -- >>> >>> Thanks & regards, >>> Nirmal >>> >>> Team Lead - WSO2 Machine Learner >>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>> Mobile: +94715779733 >>> Blog: http://nirmalfdo.blogspot.com/ >>> >>> >>> >> > -- Thanks & regards, Nirmal Team Lead - WSO2 Machine Learner Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
