[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2017-02-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15863275#comment-15863275 ] Apache Spark commented on SPARK-13450: -- User 'tejasapatil' has created a pull request for this

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2017-01-09 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15812550#comment-15812550 ] Zhan Zhang commented on SPARK-13450: ExternalAppendOnlyMap estimate the size of the data saved. In

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2016-02-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15168595#comment-15168595 ] Apache Spark commented on SPARK-13450: -- User 'shenh062326' has created a pull request for this

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2016-02-24 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15162858#comment-15162858 ] Hong Shen commented on SPARK-13450: --- Thanks, I will add in my own branch first. > SortMergeJoin will

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2016-02-23 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15160271#comment-15160271 ] Hong Shen commented on SPARK-13450: --- A join has a lot of rows with the same key. > SortMergeJoin will

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2016-02-23 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158824#comment-15158824 ] Herman van Hovell commented on SPARK-13450: --- +1 on reproduceable bugs. So you are basically

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2016-02-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158626#comment-15158626 ] Sean Owen commented on SPARK-13450: --- This isn't a helpful JIRA since you didn't say how you reproduce

[jira] [Commented] (SPARK-13450) SortMergeJoin will OOM when join rows have lot of same keys

2016-02-23 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15158566#comment-15158566 ] Hong Shen commented on SPARK-13450: --- I think we should add a ExternalAppendOnlyArrayBuffer to replace