Palamuttam [mailto:rahulpala...@gmail.com]
Sent: Thursday, September 17, 2015 3:09 PM
To: Sun, Rui
Cc: user@spark.apache.org
Subject: Re: Support of other languages?
Hi,
Thank you for both responses.
Sun you pointed out the exact issue I was referring to, which is
copying,serializing, deserializing
Hi,
Thank you for both responses.
Sun you pointed out the exact issue I was referring to, which is
copying,serializing, deserializing, the byte-array between the JVM heap and
the worker memory.
It also doesn't make sense why the byte-array should be kept on-heap, since
the data of the parent
Hi, Rahul,
To support a new language other than Java/Scala in spark, it is different
between RDD API and DataFrame API.
For RDD API:
RDD is a distributed collection of the language-specific data types whose
representation is unknown to JVM. Also transformation functions for RDD are
written
Hi Rahul,
I may not have the answer for what you are looking for but my thoughts are
given below.
I have worked with HP Vertica and R VIA UDF¹s (User Defined Functions). I
don¹t have any experience with Spark R till now. I would expect it might
follow the similar route.
UDF functions
