Hi, I have made the changes you suggested at [1] and created a API to access the wrangler service at [2].
Also added a new step in the wizard at [3] with the wrangler interface. Please have a look. How can we create a sample of the dataset to be passed to wrangler? I couldn't find any current implementation capable of this. Another concern is if we are adding the feature selection step after the cleaning step, we need to reflect the changes done in step one at step two. But since we do not apply transformations to RDD initially, we need to come up an alternative approach. [1] - https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler/src/main/java/org/wso2/carbon/ml/wrangler [2] - https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java <https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.rest.api/src/main/java/org/wso2/carbon/ml/rest/api/WranglerApiV10.java> [3] - https://github.com/danula/carbon-ml/blob/master/apps/ml/site/clean/clean.jag Thanks, Danula On Thu, Aug 27, 2015 at 9:41 AM, Danula Eranjith <hmdanu...@gmail.com> wrote: > Basically script exported from Wrangler tool has list of operations. > Wrangler class parse that script and create WranglerOperation object for > each operation with its parameters. > Then when WranglerOperation.executeOperation() is invoked, it creates the > respective SparkOperation object and then applies operations to the JavaRDD > > On Thu, Aug 27, 2015 at 9:35 AM, Nirmal Fernando <nir...@wso2.com> wrote: > >> What does WranglerOperation class do? >> >> On Thu, Aug 27, 2015 at 9:24 AM, Danula Eranjith <hmdanu...@gmail.com> >> wrote: >> >>> Currently Wrangler Operation is the class that holds details related to >>> wrangler and SparkOperation contains the relevant Spark transformation. >>> >>> If we are changing SparkOperation as WranglerOperation, we need to >>> rename the current WranglerOperation into something else. >>> >>> On Thu, Aug 27, 2015 at 9:18 AM, Nirmal Fernando <nir...@wso2.com> >>> wrote: >>> >>>> /s/SparkOpration/SparkOperation >>>> >>>> May be as Supun said, I too think we should call them as >>>> 'WranglerOperation'. >>>> >>>> On Thu, Aug 27, 2015 at 7:02 AM, Nirmal Fernando <nir...@wso2.com> >>>> wrote: >>>> >>>>> Also, avoid static methods in transformations. >>>>> >>>>> On Thu, Aug 27, 2015 at 2:48 AM, Supun Sethunga <sup...@wso2.com> >>>>> wrote: >>>>> >>>>>> Hi Danula, >>>>>> >>>>>> Few comments: >>>>>> >>>>>> - You might have to register the component in the OSGI >>>>>> environment, to be able to call the services from a another component. >>>>>> Refer [1] on how to do this. >>>>>> - Better to introduce an interface for WranglerOperation class. >>>>>> - Add class level/ method level comments. >>>>>> - Use logger in-place of System.out.println >>>>>> >>>>>> [1] >>>>>> https://github.com/danula/carbon-ml/blob/master/components/ml/org.wso2.carbon.ml.database/src/main/java/org/wso2/carbon/ml/database/internal/ds/MLDatabaseServiceDS.java >>>>>> >>>>>> Thanks, >>>>>> Supun >>>>>> >>>>>> On Wed, Aug 26, 2015 at 1:32 PM, Danula Eranjith <hmdanu...@gmail.com >>>>>> > wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I added the component at [1] >>>>>>> <https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler> >>>>>>> Please have a look. >>>>>>> >>>>>>> [1] - >>>>>>> https://github.com/danula/carbon-ml/tree/master/components/ml/org.wso2.carbon.ml.wrangler >>>>>>> >>>>>>> Danula >>>>>>> >>>>>>> On Tue, Aug 25, 2015 at 8:35 PM, Danula Eranjith < >>>>>>> hmdanu...@gmail.com> wrote: >>>>>>> >>>>>>>> Thanks Supun >>>>>>>> >>>>>>>> On Tue, Aug 25, 2015 at 7:25 PM, Supun Sethunga <sup...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> You can integrate it to [1], by adding a new component >>>>>>>>> "org.wso2.carbon.ml.wrangler". Each component is a carbon component. >>>>>>>>> >>>>>>>>> Please follow the naming conventions used in the other components, >>>>>>>>> for package names and etc.. >>>>>>>>> >>>>>>>>> [1] https://github.com/wso2/carbon-ml/tree/master/components/ml >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Supun >>>>>>>>> >>>>>>>>> On Tue, Aug 25, 2015 at 7:33 AM, Danula Eranjith < >>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> Can you suggest where I should be ideally integrating these files >>>>>>>>>> [1] >>>>>>>>>> <https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src> >>>>>>>>>> in ML. >>>>>>>>>> >>>>>>>>>> [1] - >>>>>>>>>> https://github.com/danula/wso2-ml-wrangler-integration/tree/master/src >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Danula >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Supun Sethunga* >>>>>>>>> Software Engineer >>>>>>>>> WSO2, Inc. >>>>>>>>> http://wso2.com/ >>>>>>>>> lean | enterprise | middleware >>>>>>>>> Mobile : +94 716546324 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Supun Sethunga* >>>>>> Software Engineer >>>>>> WSO2, Inc. >>>>>> http://wso2.com/ >>>>>> lean | enterprise | middleware >>>>>> Mobile : +94 716546324 >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Thanks & regards, >>>>> Nirmal >>>>> >>>>> Team Lead - WSO2 Machine Learner >>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>> Mobile: +94715779733 >>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Team Lead - WSO2 Machine Learner >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Team Lead - WSO2 Machine Learner >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> >
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
