[
https://issues.apache.org/jira/browse/SPARK-12360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15063352#comment-15063352
]
Shivaram Venkataraman commented on SPARK-12360:
-----------------------------------------------
For all the random seed cases we should just accept integers on the R side and
then convert them to long on scala (through a util function ?). This is partly
because set.seed in R takes an integer and so I assume R users are fine with
integer seeds
For the window spec functions its a big more tricky - I guess the start and end
refer to row indices here ? If so I see the concern. But I still am not sure
its a good reason to add a whole new dependency etc. A simple workaround would
be to support rangeBetween, rowsBetween which also take Strings as arguments
and then to a `.toLong` on the Scala side. So in case somebody wants to use
something bigger than an integer, they can try to use that ?
> Support using 64-bit long type in SparkR
> ----------------------------------------
>
> Key: SPARK-12360
> URL: https://issues.apache.org/jira/browse/SPARK-12360
> Project: Spark
> Issue Type: New Feature
> Components: SparkR
> Affects Versions: 1.5.2
> Reporter: Sun Rui
>
> R has no support for 64-bit integers. While in Scala/Java API, some methods
> have one or more arguments of long type. Currently we support only passing an
> integer cast from a numeric to Scala/Java side for parameters of long type of
> such methods. This may have problem covering large data sets.
> Storing a 64-bit integer in a double obviously does not work as some 64-bit
> integers can not be exactly represented in double format, so x and x+1 can't
> be distinguished.
> There is a bit64 package
> (https://cran.r-project.org/web/packages/bit64/index.html) in CRAN which
> supports vectors of 64-bit integers. We can investigate if it can be used for
> this purpose.
> two questions are:
> 1. Is the license acceptable?
> 2. This will have SparkR depends on a non-base third-party package, which
> may complicate the deployment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
