[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-10-09 Thread Michael Park (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197402#comment-16197402 ] Michael Park commented on SPARK-20589: -- Pardon my ignorance of the inner workings of

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-09-11 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161561#comment-16161561 ] Apache Spark commented on SPARK-20589: -- User 'dhruve' has created a pull request for

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-09-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157424#comment-16157424 ] Apache Spark commented on SPARK-20589: -- User 'dhruve' has created a pull request for

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144025#comment-16144025 ] Apache Spark commented on SPARK-20589: -- User 'dhruve' has created a pull request for

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-16 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129380#comment-16129380 ] Imran Rashid commented on SPARK-20589: -- Its pretty much the same thing whether you'r

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-16 Thread Amit Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129289#comment-16129289 ] Amit Kumar commented on SPARK-20589: As you said, adding job boundary via code will b

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-16 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129198#comment-16129198 ] Imran Rashid commented on SPARK-20589: -- the proposed solution will only let you cont

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127942#comment-16127942 ] Thomas Graves commented on SPARK-20589: --- Note that this type of option is also alre

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-15 Thread Amit Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127930#comment-16127930 ] Amit Kumar commented on SPARK-20589: [~imranr] Like you said, we could restrict the n

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127911#comment-16127911 ] Imran Rashid commented on SPARK-20589: -- couldn't you just limit the number of execut

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-04 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114628#comment-16114628 ] Dhruve Ashar commented on SPARK-20589: -- I have a patch for this and would like to ha

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-08-04 Thread Dhruve Ashar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16114626#comment-16114626 ] Dhruve Ashar commented on SPARK-20589: -- Spark defines stages based on the shuffle de

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-06-14 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049228#comment-16049228 ] Thomas Graves commented on SPARK-20589: --- [~Robin Shao] can you please clarify your

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-06-08 Thread Fei Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043877#comment-16043877 ] Fei Shao commented on SPARK-20589: -- Tasks are assigned to executors. If we set the numbe

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-05-24 Thread Mark Nelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16023769#comment-16023769 ] Mark Nelson commented on SPARK-20589: - I would find this very useful. We're currentl

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-05-15 Thread Amit Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011024#comment-16011024 ] Amit Kumar commented on SPARK-20589: I can probably give more reference. This origina

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-05-03 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995650#comment-15995650 ] Thomas Graves commented on SPARK-20589: --- thanks for the suggestions [~mridulm80].

[jira] [Commented] (SPARK-20589) Allow limiting task concurrency per stage

2017-05-03 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995550#comment-15995550 ] Mridul Muralidharan commented on SPARK-20589: - coalasce with shuffle=false mi