[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3652 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67885065 Cool I just tested this on a Hadoop 2.4 cluster and it worked like a charm. I'm merging this into master thanks @sryza @vanzin. --- If your project is set up for it,

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67746661 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67746658 [Test build #24676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24676/consoleFull) for PR 3652 at commit [`2791158`](https://gith

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67744200 [Test build #24676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24676/consoleFull) for PR 3652 at commit [`2791158`](https://githu

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-20 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67744080 Cool, updated patch reflects Marcelo and Andrew's comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-19 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67722849 Just took a deeper look comparing the old code with the new code and couldn't find any glaring problems. I'll test and merge this once you address the comments. ---

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-67547581 This looks fine on a first glance. I plan to take a detailed pass in the next day or two to make sure we didn't change any logic. --- If your project is set up for it

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r22067156 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -172,7 +173,7 @@ private[yarn] abstract class YarnAllocator(

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r22066707 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala --- @@ -54,15 +82,45 @@ trait YarnRMClient { * @param status The fina

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r22066685 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -498,26 +494,160 @@ private[yarn] abstract class YarnAllocator(

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r22066629 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -498,26 +494,160 @@ private[yarn] abstract class YarnAllocator(

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r22066595 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -51,13 +53,14 @@ object AllocationType extends Enumeration { //

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-66390771 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-66390767 [Test build #24273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24273/consoleFull) for PR 3652 at commit [`17628f8`](https://gith

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-66383553 I assume most here is just code motion, so I didn't pay that close attention to the changes in the container allocation code. If that's true, then this LGTM. --- If your

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r21574918 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala --- @@ -54,15 +82,45 @@ trait YarnRMClient { * @param status The final st

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r21574859 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala --- @@ -54,15 +82,45 @@ trait YarnRMClient { * @param status The final st

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r21574782 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala --- @@ -17,19 +17,33 @@ package org.apache.spark.deploy.yarn

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3652#issuecomment-66383173 [Test build #24273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24273/consoleFull) for PR 3652 at commit [`17628f8`](https://githu

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/3652#discussion_r21574615 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -17,7 +17,8 @@ package org.apache.spark.deploy.yarn

[GitHub] spark pull request: SPARK-4447. Remove layers of abstraction in YA...

2014-12-09 Thread sryza
GitHub user sryza opened a pull request: https://github.com/apache/spark/pull/3652 SPARK-4447. Remove layers of abstraction in YARN code no longer needed after dropping yarn-alpha You can merge this pull request into a Git repository by running: $ git pull https://github.com/