Actually I am free only from 12.15 am to 1.15 am. But can make myself available from 10.15 am to 12.15 am if the previous time is not feasible.
On Sun, Jun 28, 2015 at 11:21 AM, Nirmal Fernando <nir...@wso2.com> wrote: > Let us know feasible time slots for tomorrow please. > > On Sun, Jun 28, 2015 at 11:20 AM, Nirmal Fernando <nir...@wso2.com> wrote: > >> Cool, thanks. Will send an invite. >> >> On Sun, Jun 28, 2015 at 11:18 AM, Danula Eranjith <hmdanu...@gmail.com> >> wrote: >> >>> Okay Sure. >>> We can have a hangout >>> >>> On Sun, Jun 28, 2015 at 11:15 AM, Nirmal Fernando <nir...@wso2.com> >>> wrote: >>> >>>> It'll be good if we can have it before mid evaluations. If you can't >>>> make it to Trace, we can have a hangout? >>>> >>>> On Sun, Jun 28, 2015 at 11:11 AM, Danula Eranjith <hmdanu...@gmail.com> >>>> wrote: >>>> >>>>> It would be difficult for me to make it tomorrow. >>>>> How about Thursday (02/07) at Trace? anytime after 11.30 am would be >>>>> great. >>>>> >>>>> On Sun, Jun 28, 2015 at 10:09 AM, Nirmal Fernando <nir...@wso2.com> >>>>> wrote: >>>>> >>>>>> +1 shall we have it tomorrow at Trace? >>>>>> >>>>>> On Sun, Jun 28, 2015 at 9:45 AM, Supun Sethunga <sup...@wso2.com> >>>>>> wrote: >>>>>> >>>>>>> Can you arrange a time around this week? Please check with Nirmal >>>>>>> too. >>>>>>> >>>>>>> On Sun, Jun 28, 2015 at 9:31 AM, Danula Eranjith < >>>>>>> hmdanu...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> No, We haven't done a review yet. >>>>>>>> It would be great if we could have one so that I can discuss with >>>>>>>> you all and clarify the next steps of the implementation as you >>>>>>>> mentioned. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Danula >>>>>>>> >>>>>>>> On Sun, Jun 28, 2015 at 9:25 AM, Supun Sethunga <sup...@wso2.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Danula, >>>>>>>>> >>>>>>>>> Did we have a review for the work done so far? If not, shall we >>>>>>>>> have a one? We can clear out any doubts and issues as well.. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Supun >>>>>>>>> >>>>>>>>> On Wed, Jun 24, 2015 at 6:42 AM, Nirmal Fernando <nir...@wso2.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Danula, >>>>>>>>>> >>>>>>>>>> Thanks for the update, keep them coming. >>>>>>>>>> >>>>>>>>>> On a JavaRDD you can perform a collect() to get a list, AFAIR. >>>>>>>>>> Yes, this is costly, since it would load whole dataset into memory. >>>>>>>>>> So, is >>>>>>>>>> this an operation which involves multiple rows? >>>>>>>>>> >>>>>>>>>> On Tue, Jun 23, 2015 at 2:15 PM, Danula Eranjith < >>>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Supun, >>>>>>>>>>> >>>>>>>>>>> I modified the "Fill" operation to add what you mentioned. >>>>>>>>>>> >>>>>>>>>>> I used a workaround to to implement certain parts of the >>>>>>>>>>> operations such as filling with values from rows above and below. >>>>>>>>>>> I created a List Implementation using toArray() method >>>>>>>>>>> in JavaRDD and then converted it back to a JavaRDD after the >>>>>>>>>>> operation. >>>>>>>>>>> >>>>>>>>>>> This will be inefficient (in terms of both memory and time) when >>>>>>>>>>> working with very large data sets. But I think its important to >>>>>>>>>>> have these >>>>>>>>>>> features included. Otherwise a user would be left with very limited >>>>>>>>>>> set of >>>>>>>>>>> operations. >>>>>>>>>>> >>>>>>>>>>> Please let me know if you have a different opinion on this. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Danula >>>>>>>>>>> >>>>>>>>>>> On Tue, Jun 16, 2015 at 9:44 AM, Supun Sethunga <sup...@wso2.com >>>>>>>>>>> > wrote: >>>>>>>>>>> >>>>>>>>>>>> Somehow there are issues in implementing certain wrangler >>>>>>>>>>>>> functions due to limitations in JavaRDD used in spark >>>>>>>>>>>>> e.g. - >>>>>>>>>>>>> Fill operation - when filling with values from rows above and >>>>>>>>>>>>> below >>>>>>>>>>>>> Fold operation >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Agree, since rows will get executed randomly with spark, >>>>>>>>>>>> inter-row operations are not very meaningful. >>>>>>>>>>>> But you can slightly modify the implementation of the "Fill" >>>>>>>>>>>> operation, such as, to fill values based on an >>>>>>>>>>>> expression/static-value/mean >>>>>>>>>>>> etc. (not depending on other rows).. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Supun >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Jun 16, 2015 at 9:27 AM, Supun Sethunga < >>>>>>>>>>>> sup...@wso2.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Danula, >>>>>>>>>>>>> >>>>>>>>>>>>> Sorry for the late reply. Have you got the details you were >>>>>>>>>>>>> looking for? >>>>>>>>>>>>> >>>>>>>>>>>>> It would be great if I could get to know which wrangler >>>>>>>>>>>>>> operations are important for a user of the ML >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Other than the ones you have mentioned in the proposal, think >>>>>>>>>>>>> its better to have "Translate" operation as well (to create a >>>>>>>>>>>>> new column based on an existing column). >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Supun >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Thu, Jun 4, 2015 at 10:11 PM, Danula Eranjith < >>>>>>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I am currently working on generating spark transformations >>>>>>>>>>>>>> related to the operations available in the data wrangler. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Data wrangler provides sufficient parameters to re-create >>>>>>>>>>>>>> these at spark.I have successfully implemented delete and split >>>>>>>>>>>>>> operations >>>>>>>>>>>>>> of wrangler in spark. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Once this phase is completed, I can either directly generate >>>>>>>>>>>>>> these scripts at wrangler or use the javascript output and >>>>>>>>>>>>>> convert it to >>>>>>>>>>>>>> spark depending on the implementation. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Somehow there are issues in implementing certain wrangler >>>>>>>>>>>>>> functions due to limitations in JavaRDD used in spark >>>>>>>>>>>>>> >>>>>>>>>>>>>> e.g. - >>>>>>>>>>>>>> Fill operation - when filling with values from rows above and >>>>>>>>>>>>>> below >>>>>>>>>>>>>> Fold operation >>>>>>>>>>>>>> >>>>>>>>>>>>>> It would be great if I could get to know which wrangler >>>>>>>>>>>>>> operations are important for a user of the ML >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Danula >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Wed, Jun 3, 2015 at 8:30 AM, Nirmal Fernando < >>>>>>>>>>>>>> nir...@wso2.com> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Danula, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please send an update of your work thus far. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, May 10, 2015 at 2:30 PM, Nirmal Fernando < >>>>>>>>>>>>>>> nir...@wso2.com> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Danula, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Welcome to GSoC 15' ! Can you do some research on directly >>>>>>>>>>>>>>>> generating spark transformations using Wrangler and come up >>>>>>>>>>>>>>>> with a summary ? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Fri, May 8, 2015 at 11:03 AM, Danula Eranjith < >>>>>>>>>>>>>>>> hmdanu...@gmail.com> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thank you for selecting my proposal [1] >>>>>>>>>>>>>>>>> <https://docs.google.com/document/d/18NFa23CrhXqnHrkl_AuRz3sQ3Axg7SEmiA7l66Hl9_0/edit?usp=sharing> >>>>>>>>>>>>>>>>> for GSoC 2015. I am really looking forward to work with you >>>>>>>>>>>>>>>>> all and >>>>>>>>>>>>>>>>> contribute to WSO2. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I have already completed my primary research on wrangler >>>>>>>>>>>>>>>>> and would like to meet you to get feedback on the proposed >>>>>>>>>>>>>>>>> architecture. I >>>>>>>>>>>>>>>>> am planning to start working on the project before 25th of >>>>>>>>>>>>>>>>> May. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thank you, >>>>>>>>>>>>>>>>> Danula >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1] - >>>>>>>>>>>>>>>>> https://docs.google.com/document/d/18NFa23CrhXqnHrkl_AuRz3sQ3Axg7SEmiA7l66Hl9_0/edit?usp=sharing >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks & regards, >>>>>>>>>>>>>>> Nirmal >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>>>>>>> Mobile: +94715779733 >>>>>>>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> *Supun Sethunga* >>>>>>>>>>>>> Software Engineer >>>>>>>>>>>>> WSO2, Inc. >>>>>>>>>>>>> http://wso2.com/ >>>>>>>>>>>>> lean | enterprise | middleware >>>>>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> *Supun Sethunga* >>>>>>>>>>>> Software Engineer >>>>>>>>>>>> WSO2, Inc. >>>>>>>>>>>> http://wso2.com/ >>>>>>>>>>>> lean | enterprise | middleware >>>>>>>>>>>> Mobile : +94 716546324 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> Thanks & regards, >>>>>>>>>> Nirmal >>>>>>>>>> >>>>>>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>>>>>> Mobile: +94715779733 >>>>>>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> *Supun Sethunga* >>>>>>>>> Software Engineer >>>>>>>>> WSO2, Inc. >>>>>>>>> http://wso2.com/ >>>>>>>>> lean | enterprise | middleware >>>>>>>>> Mobile : +94 716546324 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> *Supun Sethunga* >>>>>>> Software Engineer >>>>>>> WSO2, Inc. >>>>>>> http://wso2.com/ >>>>>>> lean | enterprise | middleware >>>>>>> Mobile : +94 716546324 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Thanks & regards, >>>>>> Nirmal >>>>>> >>>>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>>>> Mobile: +94715779733 >>>>>> Blog: http://nirmalfdo.blogspot.com/ >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Thanks & regards, >>>> Nirmal >>>> >>>> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >>>> Mobile: +94715779733 >>>> Blog: http://nirmalfdo.blogspot.com/ >>>> >>>> >>>> >>> >> >> >> -- >> >> Thanks & regards, >> Nirmal >> >> Associate Technical Lead - Data Technologies Team, WSO2 Inc. >> Mobile: +94715779733 >> Blog: http://nirmalfdo.blogspot.com/ >> >> >> > > > -- > > Thanks & regards, > Nirmal > > Associate Technical Lead - Data Technologies Team, WSO2 Inc. > Mobile: +94715779733 > Blog: http://nirmalfdo.blogspot.com/ > > >
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
