[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-08-20 Thread Littlestar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14704534#comment-14704534 ] Littlestar commented on SPARK-2883: --- spark 1.4.1: The orc file writer relies on

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-08-20 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14705073#comment-14705073 ] Yin Huai commented on SPARK-2883: - Yeah. It is because ORC is still a part of Hive in our

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603166#comment-14603166 ] Zhan Zhang commented on SPARK-2883: --- [~philclaridge] Please refer to the test case in

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603187#comment-14603187 ] Yin Huai commented on SPARK-2883: - You can use DataFrameWriter

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Phil Claridge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603507#comment-14603507 ] Phil Claridge commented on SPARK-2883: -- Took your data frame.write.format sample and

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603164#comment-14603164 ] Zhan Zhang commented on SPARK-2883: --- [~biao luo] saveAsOrcFile and orcFile is not

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603554#comment-14603554 ] Michael Armbrust commented on SPARK-2883: - Usage questions should probably go to

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Phil Claridge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603505#comment-14603505 ] Phil Claridge commented on SPARK-2883: -- Please can you provide link to text case?

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603669#comment-14603669 ] Zhan Zhang commented on SPARK-2883: --- [~philclaridge] I try the spark-shell in local

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-26 Thread Phil Claridge (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602622#comment-14602622 ] Phil Claridge commented on SPARK-2883: -- Unable to run with Spark 1.4. Can not find

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-25 Thread biao luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601273#comment-14601273 ] biao luo commented on SPARK-2883: - peopleSchemaRDD.saveAsOrcFile(people.orc) val orcFile =

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-06-25 Thread biao luo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601275#comment-14601275 ] biao luo commented on SPARK-2883: - peopleSchemaRDD.saveAsOrcFile(people.orc) val orcFile =

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-05-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14546394#comment-14546394 ] Patrick Wendell commented on SPARK-2883: Since this is a feature I'm going to drop

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-05-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14545802#comment-14545802 ] Apache Spark commented on SPARK-2883: - User 'liancheng' has created a pull request for

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-05-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14542903#comment-14542903 ] Apache Spark commented on SPARK-2883: - User 'zhzhan' has created a pull request for

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-04-02 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393141#comment-14393141 ] Zhan Zhang commented on SPARK-2883: --- Following code demonstrate the usage of the orc

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2015-03-30 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387532#comment-14387532 ] Apache Spark commented on SPARK-2883: - User 'zhzhan' has created a pull request for

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-12-20 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14255079#comment-14255079 ] Apache Spark commented on SPARK-2883: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-11-17 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14215643#comment-14215643 ] Apache Spark commented on SPARK-2883: - User 'scwf' has created a pull request for this

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-11-03 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195301#comment-14195301 ] René X Parra commented on SPARK-2883: - [~marmbrus] I see this was changed from Version

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-11-03 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195308#comment-14195308 ] Michael Armbrust commented on SPARK-2883: - The merge deadline for 1.2.0 was on

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-11-03 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195427#comment-14195427 ] Zhan Zhang commented on SPARK-2883: --- [~neoword] , As [~marmbrus] mentioned, the PR need

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-10-15 Thread Fi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172917#comment-14172917 ] Fi commented on SPARK-2883: --- This sounds great! Is this a scala only patch, or would it also

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-10-15 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173269#comment-14173269 ] Zhan Zhang commented on SPARK-2883: --- It is scala patch, and I am not very familiar with

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-10-14 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171654#comment-14171654 ] Zhan Zhang commented on SPARK-2883: --- I almost finished the prototype, and following is

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-17 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137759#comment-14137759 ] Zhan Zhang commented on SPARK-2883: --- I am starting to prototyping the last feature with

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-16 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136102#comment-14136102 ] Zhan Zhang commented on SPARK-2883: --- There are several features to be supported. 1st:

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-16 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136407#comment-14136407 ] Michael Armbrust commented on SPARK-2883: - #1 should be easy, that information is

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136455#comment-14136455 ] Yin Huai commented on SPARK-2883: - The column projection issue is actually a bug. Because

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136478#comment-14136478 ] Yin Huai commented on SPARK-2883: - I have created

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-16 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136481#comment-14136481 ] Zhan Zhang commented on SPARK-2883: --- Sorry, I mean column pruning. Currently what I see

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-16 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136521#comment-14136521 ] Yin Huai commented on SPARK-2883: - Oh, seems types (including field names) of the

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-12 Thread Fi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132472#comment-14132472 ] Fi commented on SPARK-2883: --- I was able to run a simple query and access my ORC hive table

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-09-02 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118921#comment-14118921 ] Michael Armbrust commented on SPARK-2883: - Can you elaborate on what you mean

[jira] [Commented] (SPARK-2883) Spark Support for ORCFile format

2014-08-15 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099275#comment-14099275 ] Zhan Zhang commented on SPARK-2883: --- Spark with Hive12 can operate Orc table through