[
https://issues.apache.org/jira/browse/SPARK-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14692600#comment-14692600
]
Yu Ishikawa commented on SPARK-9427:
------------------------------------
[~shivaram] After all, I'd like to split this issue to a few sub-issues. Since
it is quite difficult to add the listed expressions at once. And since it is a
little hard to review a PR for this issue. I think we could classify them to at
least three types in SparkR. What do you think?
1. Add expressions whose parameter are only {{(Column)}} or {{(Column,
Column)}}, like {{md5(e: Column)}}
2. Add expressions whose parameter are a little complicated, like {{conv(num:
Column, fromBase: Int, toBase: Int)}}
3. Add expressions which are conflicted with the already existing generic, like
{{coalesce(e: Column*)}}
{{1}} is not a difficult task, extracting method definitions from Scala code.
And I think we rarely need to consider the confliction with current SparkR code.
However, {{2}} and {{3}} are a little hard because of the complexityomplexity.
For example, in {{3}}, if we must modify the existing R's generic due to new
expressions, we should check whether the modification affects the existing code
or not.
> Add expression functions in SparkR
> ----------------------------------
>
> Key: SPARK-9427
> URL: https://issues.apache.org/jira/browse/SPARK-9427
> Project: Spark
> Issue Type: New Feature
> Components: SparkR
> Reporter: Yu Ishikawa
>
> The list of functions to add is based on SQL's functions. And it would be
> better to add them in one shot PR.
> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/functions.scala
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
