[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-04-10 Thread Charles Pritchard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963367#comment-15963367 ] Charles Pritchard commented on SPARK-19352: --- Does this fix the issue in SPARK-18934 ? >

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-04-10 Thread Charles Pritchard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963365#comment-15963365 ] Charles Pritchard commented on SPARK-19352: --- [~cloud_fan] Yes, Hive relies on sorting

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-04-07 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961686#comment-15961686 ] Wenchen Fan commented on SPARK-19352: - I don't think Spark will provide API support for this

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-04-07 Thread Charles Pritchard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15961389#comment-15961389 ] Charles Pritchard commented on SPARK-19352: --- [~cloud_fan] Is there something on the roadmap to

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-02-24 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884032#comment-15884032 ] Wenchen Fan commented on SPARK-19352: - I'm going to mark it as `not a problem`. Spark doesn't

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-02-24 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884016#comment-15884016 ] Liang-Chi Hsieh commented on SPARK-19352: - I think this is in fact solved by SPARK-19563.

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-02-03 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15851387#comment-15851387 ] Wenchen Fan commented on SPARK-19352: - DataFrameWriter doesn't allow users to write data out orderly,

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-27 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843736#comment-15843736 ] Apache Spark commented on SPARK-19352: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-24 Thread Ivan Gozali (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836653#comment-15836653 ] Ivan Gozali commented on SPARK-19352: - Does this mean that {{Dataset.write.partitionBy()}} performs a

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

2017-01-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15836626#comment-15836626 ] Sean Owen commented on SPARK-19352: --- You repartition by userID after sorting -- is that not probably