[jira] [Commented] (SPARK-23496) Locality of coalesced partitions can be severely skewed by the order of input partitions

2018-02-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374530#comment-16374530 ] Marco Gaido commented on SPARK-23496: - [~ala.luszczak] thanks for your answer. Honestly I don't see

[jira] [Commented] (SPARK-23496) Locality of coalesced partitions can be severely skewed by the order of input partitions

2018-02-23 Thread Ala Luszczak (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374506#comment-16374506 ] Ala Luszczak commented on SPARK-23496: -- I agree that this solution is merely making the problem

[jira] [Commented] (SPARK-23496) Locality of coalesced partitions can be severely skewed by the order of input partitions

2018-02-23 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374439#comment-16374439 ] Marco Gaido commented on SPARK-23496: - I read that the proposed solution is to use random numbers

[jira] [Commented] (SPARK-23496) Locality of coalesced partitions can be severely skewed by the order of input partitions

2018-02-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374429#comment-16374429 ] Apache Spark commented on SPARK-23496: -- User 'ala' has created a pull request for this issue: