[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255424#comment-15255424 ] Apache Spark commented on SPARK-12148: -- User 'felixcheung' has created a pull request for this issue: https://github.com/apache/spark/pull/12647 > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence >Assignee: Felix Cheung > Fix For: 2.0.0 > > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254762#comment-15254762 ] Apache Spark commented on SPARK-12148: -- User 'felixcheung' has created a pull request for this issue: https://github.com/apache/spark/pull/12621 > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15249109#comment-15249109 ] Sun Rui commented on SPARK-12148: - [~felixcheung] Go ahead with this:) > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248272#comment-15248272 ] Felix Cheung commented on SPARK-12148: -- I'm up for this if [~sunrui] you haven't started > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209486#comment-15209486 ] Reynold Xin commented on SPARK-12148: - Let's just do the renaming. Thanks! > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208272#comment-15208272 ] Sun Rui commented on SPARK-12148: - Since R is dynamic language (no need to specify type when using a variable), it is supposed that renaming DataFrame to SparkDataFrame is generally an internal change and should have less impact on the existing SparkR client code (if the code does not check class name "DataFrame" somewhere. generally no need to do so. But it is possible that some client code may be broken by this change). [~shivaram], [~rxin] do you agree that we do this change for Spark 2.0? > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202376#comment-15202376 ] Anupama Joshi commented on SPARK-12148: --- Run the code in this sequence to reproduce the issue - > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Sub-task > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[
https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15200940#comment-15200940
]
Sun Rui commented on SPARK-12148:
-
An issue reported in the Spark user list may be related to this naming conflict.
{code}
countData <- matrix(1:100,ncol=4)
condition <- factor(c("A","A","B","B"))
dds <- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)
Works if i dont initialize the sparkR environment.
if I do library(SparkR) and sqlContext <- sparkRSQL.init(sc) it gives
following error
> dds <- DESeqDataSetFromMatrix(countData, as.data.frame(condition), ~
> condition)
Error in DataFrame(colData, row.names = rownames(colData)) :
cannot coerce class "data.frame" to a DataFrame
{code}
> SparkR: rename DataFrame to SparkDataFrame
> --
>
> Key: SPARK-12148
> URL: https://issues.apache.org/jira/browse/SPARK-12148
> Project: Spark
> Issue Type: Sub-task
> Components: SparkR
>Reporter: Michael Lawrence
>
> The SparkR package represents a Spark DataFrame with the class "DataFrame".
> That conflicts with the more general DataFrame class defined in the S4Vectors
> package. Would it not be more appropriate to use the name "SparkDataFrame"
> instead?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074570#comment-15074570 ] Reynold Xin commented on SPARK-12148: - +1 We should consider just renaming this to SparkDataFrame in Spark 2.0. > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Improvement > Components: SparkR >Reporter: Michael Lawrence > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051720#comment-15051720 ] Michael Lawrence commented on SPARK-12148: -- I agree. A deprecation cycle is good idea. > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Wish > Components: R, SparkR >Reporter: Michael Lawrence >Priority: Minor > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051716#comment-15051716 ] Dan Putler commented on SPARK-12148: Michael Lawrence's arguments are very valid. The S4Vector package seems to be an important Bioconductor package, and likely has a lot of users in the bioinformatics community, which is a community that is also likely to have a high share of SparkR users, so the name collision issues are real. The one thing that needs to be thought through is how to mitigate the effect on existing SparkR code that users have written (I'm less concerned about the inconsistencies in the naming convention across Scala, Python, and R). Given the fairly short period of time SparkR has supported DataFrames, the amount of existing user code is likely not enormous. However, I think it does make sense to have a transition period of one Spark release where a call to (SparkR) DataFrame results in a warning that the function is being deprecated, and that SparkDataFrame (or whatever else we choose to rename it) should be used instead. > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Wish > Components: R, SparkR >Reporter: Michael Lawrence >Priority: Minor > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15043625#comment-15043625 ] Michael Lawrence commented on SPARK-12148: -- Not only would the constructor symbol conflict be highly annoying (R users have little patience for such conflicts as they are typically programming interactively), but there would also be ambiguity when specifying method signatures, inheritance relationships, etc in packages bridging the frameworks. While I can appreciate your desire for consistency, please realize that the name "data frame" is deeply rooted in R, so I'm not sure it's realistic to come into R and expect to use that name without qualification. The S4Vector DataFrame is the S4 analog of the base data.frame, with the name adjusted to conform to S4 class naming conventions. Interfaces to external systems should qualify somehow. While the name SparkDataFrame would break consistency with other interfaces, it would have the benefit of clarity, since data frame is such an overloaded term in R. > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Wish > Components: R, SparkR >Reporter: Michael Lawrence >Priority: Minor > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12148) SparkR: rename DataFrame to SparkDataFrame
[ https://issues.apache.org/jira/browse/SPARK-12148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15043591#comment-15043591 ] Shivaram Venkataraman commented on SPARK-12148: --- Hmm thats unfortunate but I think we want to keep the DataFrame name in SparkR to match the class names in Scala [1] and Python [2]. Couldn't users can always prefix their calls with SparkR:: or S4Vectors:: or does this cause some other problems at package load time ? cc [~rxin] [1] http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrame [2] http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame > SparkR: rename DataFrame to SparkDataFrame > -- > > Key: SPARK-12148 > URL: https://issues.apache.org/jira/browse/SPARK-12148 > Project: Spark > Issue Type: Wish > Components: R, SparkR >Reporter: Michael Lawrence >Priority: Minor > > The SparkR package represents a Spark DataFrame with the class "DataFrame". > That conflicts with the more general DataFrame class defined in the S4Vectors > package. Would it not be more appropriate to use the name "SparkDataFrame" > instead? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
