[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19534 @sitalkedia would you please reopen this PR, I think the second issue I fixed before is not valid anymore, for the first issue the fix is no difference compared to here. ---

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19534 Let's fix up Saisai's PR then. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-22 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19534 @sitalkedia I'm OK with either. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-20 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19534 I think other PR is fixing one more issue on top of runningTasks being negative, so we can proceed with the other one. What do you think @jerryshao ? ---

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-20 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19534 @sitalkedia That makes sense. The proposed solutions are quite similar, we can choose to continue with either PR, WDYT @jerryshao @sitalkedia ? ---

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19534 @sitalkedia I have a very old similar PR #11205 , maybe you can refer to it. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19534 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82914/ Test PASSed. ---

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19534 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19534 **[Test build #82914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82914/testReport)** for PR 19534 at commit

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19534 **[Test build #82914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82914/testReport)** for PR 19534 at commit

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19534 Jenkins retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19534 @jiangxb1987 - yes that is the issue and you are right, we can avoid it by checking if the stageId is valid when we get a task end event. But I like this approach better because we can clean up

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19534 Do you mean we may first set `numRunningTasks` to 0 and then run into `onTaskEnd` and have `numRunningTasks -= 1`? Could we simply check `stageIdToSpeculativeTaskIndices`/`stageIdToTaskIndices`

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19534 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19534 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82903/ Test FAILed. ---

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19534 **[Test build #82903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82903/testReport)** for PR 19534 at commit

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-19 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19534 @sitalkedia would you please fix the PR title, seems it is broken now. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19534 **[Test build #82903 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82903/testReport)** for PR 19534 at commit

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-18 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19534 cc - @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: