[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-07-30 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080385#comment-14080385 ] Matei Zaharia commented on SPARK-983: - Now that an ExternalSorter class from SPARK-2045

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-06-14 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031756#comment-14031756 ] Madhu Siddalingaiah commented on SPARK-983: --- Thanks Aaron. PR submitted:

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-06-12 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029498#comment-14029498 ] Aaron Davidson commented on SPARK-983: -- The idea for SizeTrackingAppendOnlyMap is that

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-06-11 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028715#comment-14028715 ] Madhu Siddalingaiah commented on SPARK-983: --- [Aaron

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-06-02 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015671#comment-14015671 ] Mark Hamstra commented on SPARK-983: Is that code visible someplace? In broad outline,

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-06-02 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015907#comment-14015907 ] Madhu Siddalingaiah commented on SPARK-983: --- Yes, the code is checked in here:

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-06-01 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14015020#comment-14015020 ] Madhu Siddalingaiah commented on SPARK-983: --- I tested some additions locally that

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-27 Thread Mark Hamstra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009875#comment-14009875 ] Mark Hamstra commented on SPARK-983: I'm hoping these can be kept orthogonal, but I

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-27 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010182#comment-14010182 ] Madhu Siddalingaiah commented on SPARK-983: --- Understood. Here are my thoughts: I

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-27 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010190#comment-14010190 ] Aaron Davidson commented on SPARK-983: -- Does sound reasonable. For some reason it does

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-26 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009145#comment-14009145 ] Patrick Wendell commented on SPARK-983: --- We are actually looking at this problem in a

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-25 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008440#comment-14008440 ] Andrew Ash commented on SPARK-983: -- Because ExternalAppendOnlyMap uses

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-25 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008554#comment-14008554 ] Aaron Davidson commented on SPARK-983: -- Historically, we have not used

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-25 Thread Aaron Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008557#comment-14008557 ] Aaron Davidson commented on SPARK-983: -- [~pwendell] or [~matei], any opinions on

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-23 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007479#comment-14007479 ] Madhu Siddalingaiah commented on SPARK-983: --- I have the beginnings of a

[jira] [Commented] (SPARK-983) Support external sorting for RDD#sortByKey()

2014-05-22 Thread Madhu Siddalingaiah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006664#comment-14006664 ] Madhu Siddalingaiah commented on SPARK-983: --- Looking at