[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-07-21 Thread ryan-williams
Github user ryan-williams commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-123487287 I've lost state on this, closing, thanks all --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-07-21 Thread ryan-williams
Github user ryan-williams closed the pull request at: https://github.com/apache/spark/pull/3525 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-06-29 Thread ilganeli
Github user ilganeli commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-116662486 @srowen @JoshRosen I think this should be refactored to use the updates from #5574 but I don't think #5574 resolves this on its own because of the need to handle the

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-06-29 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-116792614 @ryan-williams would you mind opening a new PR to reflect the changes in #5574? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-06-29 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-116536930 I think this would have to be reworked in light of SPARK-5932. I don't think that resolved this particular suggestion. Proceed that way or close this PR? --- If your

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-06-24 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-115027905 I think that this PR's motivation may have been addressed by #5574. @ilganeli @srowen @ryan-williams , can you confirm whether there's still an issue here and

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-25 Thread hammer
Github user hammer commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-76057244 @srowen this one ready to go in? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-76058874 @hammer @ryan-williams I'd prefer to hear from @JoshRosen or @pwendell before merging. The two key questions are - Gut feeling about the potential for merge

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-12 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24633577 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24632812 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24400754 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24401098 --- Diff: docs/running-on-yarn.md --- @@ -113,9 +113,9 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /tr tr

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24400791 --- Diff: core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala --- @@ -20,6 +20,7 @@ package org.apache.spark.scheduler.local import

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24401004 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -980,33 +980,77 @@ private[spark] object Utils extends Logging { ) }

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-73674272 I personally definitely like the clarity this is adding in the code. Looks like it needs another rebase and test to see where we stand with Jenkins and Mima. --- If

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-73783538 OK I think it is back to you @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24450366 --- Diff: docs/running-on-yarn.md --- @@ -113,9 +113,9 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /tr

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24450496 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -980,33 +980,77 @@ private[spark] object Utils extends Logging { ) }

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24449997 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -980,33 +980,77 @@ private[spark] object Utils extends Logging { )

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-73783554 (thanks for the review!) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24446161 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24448450 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread ryan-williams
Github user ryan-williams commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24446286 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-10 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3525#discussion_r24448190 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -203,6 +203,28 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging

[GitHub] spark pull request: [SPARK-4666] Improve YarnAllocator's parsing o...

2015-02-09 Thread ryan-williams
Github user ryan-williams commented on the pull request: https://github.com/apache/spark/pull/3525#issuecomment-73625367 OK, this is ready to go again. Changes: * removed SPARK-4665 from the title and closed that JIRA * in the process of rebasing this change I found myself