[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39191333 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -504,13 +504,36 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39198206 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -390,7 +390,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39199084 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -390,7 +390,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39198825 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -474,18 +502,31 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread tnachen
Github user tnachen commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39194384 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -390,7 +390,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139344775 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139344740 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139344890 OK, let's merge this and fix the follow-up issues separately. Thanks for your work @mccheah. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8007 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139345536 [Test build #42286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42286/consoleFull) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139386578 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139386577 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-10 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139386507 [Test build #42286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42286/console) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39112877 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -504,13 +504,36 @@ class TaskSetManagerSuite extends SparkFunSuite

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139083715 Latest change does everything sans the unit test addition with my question about it. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139084339 [Test build #42226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42226/consoleFull) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139084000 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139084020 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139110866 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139110865 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139110820 [Test build #42226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42226/console) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096167 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -709,6 +709,12 @@ private[spark] class TaskSetManager( }

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096180 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -709,6 +709,12 @@ private[spark] class TaskSetManager( }

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096233 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -194,6 +196,12 @@ case object TaskKilled extends TaskFailedReason { case

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096192 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -709,6 +709,12 @@ private[spark] class TaskSetManager( }

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096279 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -722,10 +728,7 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096250 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -332,7 +332,8 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096484 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -809,9 +814,12 @@ private[spark] class TaskSetManager( }

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39096979 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -90,6 +93,41 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39097288 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -90,6 +93,41 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39097379 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -90,6 +93,41 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39097818 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -101,6 +139,32 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39097847 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -113,6 +177,7 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39098055 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -211,6 +218,9 @@ private[yarn] class YarnAllocator( * Deal

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39098375 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -474,18 +502,31 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39098596 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -474,18 +502,31 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39098502 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -474,18 +502,31 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39098785 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -577,8 +577,10 @@ class JsonProtocolSuite extends SparkFunSuite {

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r39098952 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -504,13 +504,36 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-139052729 @mccheah @vanzin I think for this patch we can proceed without fixing the race condition. It may actually turn out to be something that's important to fix, since the

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-04 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137842134 I'm still a little nervous that automated testing wasn't really done / isn't really possible for this code, at least on an end-to-end level. Anyone have any suggestions

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137538472 re: the new race, I think it's fine to do it as a separate change. It's separate enough from the code being changed here that trying to tack both changes here would be

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137535405 The discussion was concluded that "I should do something about it". Let's not merge it until we have addressed that. I don't know if I'll have the bandwidth to do a

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137593808 [Test build #41985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41985/consoleFull) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137592096 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137592133 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137617021 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137617023 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-03 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137616955 [Test build #41985 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41985/console) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600012 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -709,6 +709,12 @@ private[spark] class TaskSetManager( }

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600072 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -31,14 +31,18 @@ import org.apache.spark.util.Utils * old stages to be

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600038 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -722,10 +728,7 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599699 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -202,8 +209,17 @@ case class TaskCommitDenied(jobID: Int, partitionID: Int,

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600287 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala --- @@ -92,6 +94,11 @@ private[spark] object

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600407 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala --- @@ -135,11 +135,11 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600357 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -264,9 +266,13 @@ class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137295040 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137294983 Accidentally missed one of @andrewor14 's comments but everything else is addressed I think. One point to note: I moved the loss-reason-request-reply logic into

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137295407 Ah sorry @mccheah I was actually not done yet. I will resume now. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137297119 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605830 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -795,7 +796,9 @@ private[spark] object JsonProtocol { case

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605840 --- Diff: core/src/test/scala/org/apache/spark/PartitioningSuite.scala --- @@ -17,13 +17,12 @@ package org.apache.spark -import

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137297485 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137297486 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137297480 [Test build #41956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41956/console) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599322 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -31,14 +31,18 @@ import org.apache.spark.util.Utils * old stages to be

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599389 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -31,14 +31,18 @@ import org.apache.spark.util.Utils * old stages to be

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599409 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -194,6 +200,7 @@ case object TaskKilled extends TaskFailedReason { case

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599854 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ExecutorLossReason.scala --- @@ -23,13 +23,19 @@ import

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599910 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ExecutorLossReason.scala --- @@ -23,13 +23,19 @@ import

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38600527 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -264,9 +266,13 @@ class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599463 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -194,6 +200,7 @@ case object TaskKilled extends TaskFailedReason { case

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38599567 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -202,8 +209,17 @@ case class TaskCommitDenied(jobID: Int, partitionID: Int,

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137295051 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137296635 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137296602 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137296753 [Test build #41956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41956/consoleFull) for PR 8007 at commit

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605550 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -91,6 +94,36 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605520 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -91,6 +94,36 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8007#issuecomment-137297118 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605811 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala --- @@ -390,7 +390,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605816 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -795,7 +796,9 @@ private[spark] object JsonProtocol { case

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38605954 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -91,6 +94,36 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38606027 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -101,6 +134,32 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607202 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -64,7 +66,7 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607194 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -67,6 +68,11 @@ private[spark] class ApplicationMaster(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607225 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -17,6 +17,7 @@ package org.apache.spark.deploy.yarn

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607757 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -91,6 +94,36 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38608209 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -211,8 +218,11 @@ private[yarn] class YarnAllocator( * Deal

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38608220 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -236,10 +246,14 @@ private[yarn] class YarnAllocator( if

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607588 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -440,22 +454,43 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607562 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -101,6 +134,32 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38608268 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -474,18 +509,26 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38608264 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -474,18 +509,26 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38612598 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -101,6 +134,32 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38612546 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -91,6 +94,36 @@ private[spark] abstract class

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607396 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -440,22 +454,43 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607365 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -440,22 +454,43 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607674 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -440,22 +454,43 @@ private[yarn] class YarnAllocator(

[GitHub] spark pull request: [SPARK-8167] Make tasks that fail from YARN pr...

2015-09-02 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/8007#discussion_r38607895 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -440,22 +454,43 @@ private[yarn] class YarnAllocator(

  1   2   3   4   >