[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-03-21 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205279#comment-15205279 ] Thomas Graves commented on SPARK-1239: -- I do like the idea of broadcast and originally when I had

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-03-21 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204939#comment-15204939 ] Mridul Muralidharan commented on SPARK-1239: [~tgraves] For the last part (waiting bit) - why

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-03-21 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15204900#comment-15204900 ] Thomas Graves commented on SPARK-1239: -- So I have been looking at this and testing a few changes out.

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2016-02-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15130351#comment-15130351 ] Daniel Darabos commented on SPARK-1239: --- I've read an interesting article about the "Kylix"

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-12-02 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036341#comment-15036341 ] Thomas Graves commented on SPARK-1239: -- I have another user hitting this also. The above mentions

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-11-04 Thread Alex Slusarenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14989638#comment-14989638 ] Alex Slusarenko commented on SPARK-1239: > How is this not a major issue for everyone? Daniel, I

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-11-03 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14987046#comment-14987046 ] Daniel Darabos commented on SPARK-1239: --- I can also add some data. I have a ShuffleMapStage with

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-07-29 Thread Alex Slusarenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14645913#comment-14645913 ] Alex Slusarenko commented on SPARK-1239: Hi, all. We have faced this issue many

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-03-08 Thread Kostas Sakellis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352306#comment-14352306 ] Kostas Sakellis commented on SPARK-1239: How many reduce side tasks do you have?

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-03-08 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352356#comment-14352356 ] Patrick Wendell commented on SPARK-1239: It would be helpful if any users who have

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-03-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352377#comment-14352377 ] Mridul Muralidharan commented on SPARK-1239: Hitting akka framesize for map

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2015-03-08 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352284#comment-14352284 ] Mridul Muralidharan commented on SPARK-1239: [~pwendell] Is there any update

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-10-23 Thread Kostas Sakellis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182110#comment-14182110 ] Kostas Sakellis commented on SPARK-1239: Apologies for not commenting on this JIRA

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-10-23 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14182122#comment-14182122 ] Patrick Wendell commented on SPARK-1239: Hey Kostas - there are a few other bugs

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-10-08 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163423#comment-14163423 ] DB Tsai commented on SPARK-1239: +1, we run into this issue as well. Don't fetch all map

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134160#comment-14134160 ] Andrew Ash commented on SPARK-1239: --- For large statuses, would we expect that to exceed

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134198#comment-14134198 ] Patrick Wendell commented on SPARK-1239: Yes, the current state of the art is to

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-08 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14126215#comment-14126215 ] Andrew Or commented on SPARK-1239: -- I have reassigned it to you Kostas. Don't fetch all

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-05 Thread Kostas Sakellis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14123883#comment-14123883 ] Kostas Sakellis commented on SPARK-1239: [~pwendell] I'd like to take a crack at

[jira] [Commented] (SPARK-1239) Don't fetch all map output statuses at each reducer during shuffles

2014-09-01 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117914#comment-14117914 ] Patrick Wendell commented on SPARK-1239: I think [~andrewor] has this in his