[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2016-10-10 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564071#comment-15564071 ] holdenk commented on SPARK-4630: I also agree this would be really good to revisit, from t

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2016-10-10 Thread Mike Dusenberry (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563839#comment-15563839 ] Mike Dusenberry commented on SPARK-4630: It would be really nice to revisit this i

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-20 Thread Kostas Sakellis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14284341#comment-14284341 ] Kostas Sakellis commented on SPARK-4630: I agree that this should be build not ass

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-20 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283809#comment-14283809 ] Lianhui Wang commented on SPARK-4630: - I think it is better that we use stage's output

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283539#comment-14283539 ] Reynold Xin commented on SPARK-4630: Maybe the first step towards this is to find what

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-19 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283225#comment-14283225 ] Sandy Ryza commented on SPARK-4630: --- One way I was thinking it might make sense to accom

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283161#comment-14283161 ] Reynold Xin commented on SPARK-4630: The problem is that as the stage if producing its

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-19 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283146#comment-14283146 ] Sandy Ryza commented on SPARK-4630: --- [~rxin] I agree that there are probably a ton of pl

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282273#comment-14282273 ] Reynold Xin commented on SPARK-4630: This seems like something better suited for Schem

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2015-01-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14280173#comment-14280173 ] Apache Spark commented on SPARK-4630: - User 'lianhuiwang' has created a pull request f

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-30 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229407#comment-14229407 ] Patrick Wendell commented on SPARK-4630: Thanks Sandy - that's useful context. I t

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-30 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229371#comment-14229371 ] Sandy Ryza commented on SPARK-4630: --- Hey [~pwendell], Spark deals much better with large

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-29 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229001#comment-14229001 ] Patrick Wendell commented on SPARK-4630: Hey Kos - before starting to work on the

[jira] [Commented] (SPARK-4630) Dynamically determine optimal number of partitions

2014-11-26 Thread Lianhui Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227280#comment-14227280 ] Lianhui Wang commented on SPARK-4630: - i am interesting in this. and we can learn some