t; Thanks Hyukjin! Very cool results
>>
>> Shivaram
>> On Fri, Nov 9, 2018 at 10:58 AM Felix Cheung
>> wrote:
>> >
>> > Very cool!
>> >
>> >
>> > ____
>> > From: Hyukjin Kwon
>> > Sent
ix Cheung
> wrote:
> >
> > Very cool!
> >
> >
> >
> > From: Hyukjin Kwon
> > Sent: Thursday, November 8, 2018 10:29 AM
> > To: dev
> > Subject: Arrow optimization in conversion from R DataFrame to Spark
> DataFrame
&g
Thanks Hyukjin! Very cool results
Shivaram
On Fri, Nov 9, 2018 at 10:58 AM Felix Cheung wrote:
>
> Very cool!
>
>
>
> From: Hyukjin Kwon
> Sent: Thursday, November 8, 2018 10:29 AM
> To: dev
> Subject: Arrow optimization in conversio
Very cool!
From: Hyukjin Kwon
Sent: Thursday, November 8, 2018 10:29 AM
To: dev
Subject: Arrow optimization in conversion from R DataFrame to Spark DataFrame
Hi all,
I am trying to introduce R Arrow optimization by reusing PySpark Arrow
optimization
Hi all,
I am trying to introduce R Arrow optimization by reusing PySpark Arrow
optimization.
It boosts R DataFrame > Spark DataFrame up to roughly 900% ~ 1200% faster.
Looks working fine so far; however, I would appreciate if you guys have
some time to take a look
