Great work Hyukjin! I'm not too familiar with R, but I'll take a look at the PR.
Bryan On Fri, Nov 9, 2018 at 9:19 AM Shivaram Venkataraman < shiva...@eecs.berkeley.edu> wrote: > Thanks Hyukjin! Very cool results > > Shivaram > On Fri, Nov 9, 2018 at 10:58 AM Felix Cheung <felixcheun...@hotmail.com> > wrote: > > > > Very cool! > > > > > > ________________________________ > > From: Hyukjin Kwon <gurwls...@gmail.com> > > Sent: Thursday, November 8, 2018 10:29 AM > > To: dev > > Subject: Arrow optimization in conversion from R DataFrame to Spark > DataFrame > > > > Hi all, > > > > I am trying to introduce R Arrow optimization by reusing PySpark Arrow > optimization. > > > > It boosts R DataFrame > Spark DataFrame up to roughly 900% ~ 1200% > faster. > > > > Looks working fine so far; however, I would appreciate if you guys have > some time to take a look (https://github.com/apache/spark/pull/22954) so > that we can directly go ahead as soon as R API of Arrow is released. > > > > More importantly, I want some more people who're more into Arrow R API > side but also interested in Spark side. I have already cc'ed some people I > know but please come, review and discuss for both Spark side and Arrow side. > > > > Thanks. > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
