Hi all, I am currently working on generating spark transformations related to the operations available in the data wrangler.
Data wrangler provides sufficient parameters to re-create these at spark.I have successfully implemented delete and split operations of wrangler in spark. Once this phase is completed, I can either directly generate these scripts at wrangler or use the javascript output and convert it to spark depending on the implementation. Somehow there are issues in implementing certain wrangler functions due to limitations in JavaRDD used in spark e.g. - Fill operation - when filling with values from rows above and below Fold operation It would be great if I could get to know which wrangler operations are important for a user of the ML Thanks, Danula On Wed, Jun 3, 2015 at 8:30 AM, Nirmal Fernando <nir...@wso2.com> wrote: > Hi Danula, > > Please send an update of your work thus far. > > On Sun, May 10, 2015 at 2:30 PM, Nirmal Fernando <nir...@wso2.com> wrote: > >> Hi Danula, >> >> Welcome to GSoC 15' ! Can you do some research on directly generating >> spark transformations using Wrangler and come up with a summary ? >> >> On Fri, May 8, 2015 at 11:03 AM, Danula Eranjith <hmdanu...@gmail.com> >> wrote: >> >>> Hi all, >>> >>> Thank you for selecting my proposal [1] >>> <https://docs.google.com/document/d/18NFa23CrhXqnHrkl_AuRz3sQ3Axg7SEmiA7l66Hl9_0/edit?usp=sharing> >>> for GSoC 2015. I am really looking forward to work with you all and >>> contribute to WSO2. >>> >>> I have already completed my primary research on wrangler and would like >>> to meet you to get feedback on the proposed architecture. I am planning to >>> start working on the project before 25th of May. >>> >>> Thank you, >>> Danula >>> >>> [1] - >>> https://docs.google.com/document/d/18NFa23CrhXqnHrkl_AuRz3sQ3Axg7SEmiA7l66Hl9_0/edit?usp=sharing >>> >> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > > Thanks & regards, > Nirmal > > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > >
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
