[jira] [Commented] (SPARK-12144) Implement DataFrameReader and DataFrameWriter API in SparkR

2015-12-07 Thread Sun Rui (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046376#comment-15046376
 ] 

Sun Rui commented on SPARK-12144:
-

[~yanboliang] go ahead. Could you  please change the title of this JIRA 
according to the discussion, and summarize what to be done in the description 
of this JIRA (for example, add read.parquet(), ...)? If needed, you can break 
this one into a number of sub-tasks.

> Implement DataFrameReader and DataFrameWriter API in SparkR
> ---
>
> Key: SPARK-12144
> URL: https://issues.apache.org/jira/browse/SPARK-12144
> Project: Spark
>  Issue Type: New Feature
>  Components: SparkR
>Affects Versions: 1.5.2
>Reporter: Sun Rui
>
> DataFrameReader API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader
> DataFrameWriter API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameWriter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-12144) Implement DataFrameReader and DataFrameWriter API in SparkR

2015-12-07 Thread Yanbo Liang (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15046311#comment-15046311
 ] 

Yanbo Liang commented on SPARK-12144:
-

I'm working on SPARK-12146 which is similar with this one. I can also take this 
if no one worked on it.

> Implement DataFrameReader and DataFrameWriter API in SparkR
> ---
>
> Key: SPARK-12144
> URL: https://issues.apache.org/jira/browse/SPARK-12144
> Project: Spark
>  Issue Type: New Feature
>  Components: SparkR
>Affects Versions: 1.5.2
>Reporter: Sun Rui
>
> DataFrameReader API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader
> DataFrameWriter API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameWriter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-12144) Implement DataFrameReader and DataFrameWriter API in SparkR

2015-12-04 Thread Felix Cheung (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042103#comment-15042103
 ] 

Felix Cheung commented on SPARK-12144:
--

+1 [~shivaram]

The style
{code}
read.format("json").option(options).load("people.json")
{code}

doesn't really fit R

instead, why don't we have
{code}
read(format, options_named_list, ...)
{code}

in fact we could leave/add convenience functions like
{code}
read.csv(options)
read.json(options)
read.parquet(options)
{code}



> Implement DataFrameReader and DataFrameWriter API in SparkR
> ---
>
> Key: SPARK-12144
> URL: https://issues.apache.org/jira/browse/SPARK-12144
> Project: Spark
>  Issue Type: New Feature
>  Components: SparkR
>Affects Versions: 1.5.2
>Reporter: Sun Rui
>
> DataFrameReader API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader
> DataFrameWriter API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameWriter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-12144) Implement DataFrameReader and DataFrameWriter API in SparkR

2015-12-03 Thread Sun Rui (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15041177#comment-15041177
 ] 

Sun Rui commented on SPARK-12144:
-

@shivaram, your opinion is reasonable. It seems that read.df and write.df does 
not cover all functionalities exposed by DataFrameReader and DataFrameWriter. 
Maybe we don't need to provide these two API, but provide more wrapping 
functions in SparkR.

> Implement DataFrameReader and DataFrameWriter API in SparkR
> ---
>
> Key: SPARK-12144
> URL: https://issues.apache.org/jira/browse/SPARK-12144
> Project: Spark
>  Issue Type: New Feature
>  Components: SparkR
>Affects Versions: 1.5.2
>Reporter: Sun Rui
>
> DataFrameReader API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader
> DataFrameWriter API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameWriter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Commented] (SPARK-12144) Implement DataFrameReader and DataFrameWriter API in SparkR

2015-12-03 Thread Shivaram Venkataraman (JIRA) 

[ 
https://issues.apache.org/jira/browse/SPARK-12144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15040773#comment-15040773
 ] 

Shivaram Venkataraman commented on SPARK-12144:
---

Personally I don't think this API is a good fit for SparkR -- This introduces a 
lot of chaining based methods which are awkward to write in R without using 
something like magrittr. I think the existing `read.df` and having `option = 
value` is more user-friendly.  Is there any functionality we will gain from 
this ?

> Implement DataFrameReader and DataFrameWriter API in SparkR
> ---
>
> Key: SPARK-12144
> URL: https://issues.apache.org/jira/browse/SPARK-12144
> Project: Spark
>  Issue Type: New Feature
>  Components: SparkR
>Affects Versions: 1.5.2
>Reporter: Sun Rui
>
> DataFrameReader API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameReader
> DataFrameWriter API: 
> http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.DataFrameWriter



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org