[jira] [Commented] (SPARK-15545) R remove non-exported unused methods, like jsonRDD
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322039#comment-15322039 ] Miao Wang commented on SPARK-15545: --- [~shivaram]Thanks for your clarification. I will start working on this one and submit PR after 2.0 release. By the way, can you take a look at the following PR when you have time? It is fairly simple. Thanks! https://github.com/apache/spark/pull/13508 > R remove non-exported unused methods, like jsonRDD > -- > > Key: SPARK-15545 > URL: https://issues.apache.org/jira/browse/SPARK-15545 > Project: Spark > Issue Type: Bug > Components: SparkR >Reporter: Felix Cheung >Priority: Minor > > Need to review what should be removed. > one reason to not remove this right away is because we have been talking > about calling internal methods via `SparkR:::jsonRDD` for this and other RDD > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15545) R remove non-exported unused methods, like jsonRDD
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315991#comment-15315991 ] Shivaram Venkataraman commented on SPARK-15545: --- That discussion was before we had UDFs implemented in dapply, gapply etc. I think its fine to start working on this. But we might not pull this to 2.0 and will probably include the removals in 2.1.0 (it'll also give us a chance to announce their deprecation with the 2.0.0 release) > R remove non-exported unused methods, like jsonRDD > -- > > Key: SPARK-15545 > URL: https://issues.apache.org/jira/browse/SPARK-15545 > Project: Spark > Issue Type: Bug > Components: SparkR >Reporter: Felix Cheung >Priority: Minor > > Need to review what should be removed. > one reason to not remove this right away is because we have been talking > about calling internal methods via `SparkR:::jsonRDD` for this and other RDD > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15545) R remove non-exported unused methods, like jsonRDD
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315138#comment-15315138 ] Miao Wang commented on SPARK-15545: --- Yes. All RDD APIs are not exported. According to the above PR, I have to wait for the decision before working on this one. Right? I will open a different one to export is.nan. I saw there is is.NaN there. > R remove non-exported unused methods, like jsonRDD > -- > > Key: SPARK-15545 > URL: https://issues.apache.org/jira/browse/SPARK-15545 > Project: Spark > Issue Type: Bug > Components: SparkR >Reporter: Felix Cheung >Priority: Minor > > Need to review what should be removed. > one reason to not remove this right away is because we have been talking > about calling internal methods via `SparkR:::jsonRDD` for this and other RDD > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15545) R remove non-exported unused methods, like jsonRDD
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315038#comment-15315038 ] Felix Cheung commented on SPARK-15545: -- I think that's mostly right - the RDD related methods are not exported. However, to be clear, the last time this was brought up (https://issues.apache.org/jira/browse/SPARK-12172) it was decided we should not be removing these just yet. (not sure why is.nan / isnan is not exported, but it probably should) > R remove non-exported unused methods, like jsonRDD > -- > > Key: SPARK-15545 > URL: https://issues.apache.org/jira/browse/SPARK-15545 > Project: Spark > Issue Type: Bug > Components: SparkR >Reporter: Felix Cheung >Priority: Minor > > Need to review what should be removed. > one reason to not remove this right away is because we have been talking > about calling internal methods via `SparkR:::jsonRDD` for this and other RDD > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15545) R remove non-exported unused methods, like jsonRDD
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314416#comment-15314416 ] Miao Wang commented on SPARK-15545: --- I did a search on all R files with setMethods and compare with NAMESPACE file. I have the following findings. Dataframe.R: toJSON toRDD map flatMap mapPartitions foreachPartition SQLContext.R toDF jsonRDD context.R: Only spark.lapply and setLogLevel are exported functions.R: is.nan isnan [~felixcheung] Can you review which ones from the above list should be removed? Thanks! > R remove non-exported unused methods, like jsonRDD > -- > > Key: SPARK-15545 > URL: https://issues.apache.org/jira/browse/SPARK-15545 > Project: Spark > Issue Type: Bug > Components: SparkR >Reporter: Felix Cheung >Priority: Minor > > Need to review what should be removed. > one reason to not remove this right away is because we have been talking > about calling internal methods via `SparkR:::jsonRDD` for this and other RDD > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15545) R remove non-exported unused methods, like jsonRDD
[ https://issues.apache.org/jira/browse/SPARK-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308892#comment-15308892 ] Miao Wang commented on SPARK-15545: --- I will give a try. Thanks > R remove non-exported unused methods, like jsonRDD > -- > > Key: SPARK-15545 > URL: https://issues.apache.org/jira/browse/SPARK-15545 > Project: Spark > Issue Type: Bug > Components: SparkR >Reporter: Felix Cheung >Priority: Minor > > Need to review what should be removed. > one reason to not remove this right away is because we have been talking > about calling internal methods via `SparkR:::jsonRDD` for this and other RDD > methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
