[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-28 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-143843978 The concept looks sound to me, however I'm actually less familiar with the logic of where tasks get allocated. Someone else should also review! --- If your project is s

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-28 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8887#discussion_r40588094 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -362,8 +395,8 @@ private[spark] class TaskSchedulerImpl( }

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-143832319 can you add `[YARN]` to the title? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-143751726 /cc @squito also --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142769028 [Test build #42927 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42927/console) for PR 8887 at commit [`5b90150`](https://github.

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142769080 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142769079 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142763962 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142763964 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142763862 [Test build #42923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42923/console) for PR 8887 at commit [`5b90150`](https://github.

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142743669 [Test build #42927 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42927/consoleFull) for PR 8887 at commit [`5b90150`](https://gith

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142741001 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142740970 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142740144 hmm. retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142737548 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142737544 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142735445 [Test build #42923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42923/consoleFull) for PR 8887 at commit [`5b90150`](https://gith

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142734894 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142734913 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142733640 cc'ing a few people: @mccheah (who wrote the original preemption code), @JoshRosen @andrewor14 (not sure who between you guys is reviewing this area these days).

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142732529 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-142732555 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10622] [core] Differentiate dead from "...

2015-09-23 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/8887 [SPARK-10622] [core] Differentiate dead from "mostly dead" executors. In YARN mode, when preemption is enabled, we may leave executors in a zombie state while we wait to retrieve the reason for wh