Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-45445910
I've merged this - thanks @cmccabe definitely a good improvement!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub a
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/850
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabl
Github user cmccabe commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44564644
I tested this by running Spark on YARN against Hadoop 2.4 on a 4 node
cluster. I used the Pi example job.
---
If your project is set up for it, you can reply to this ema
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44493998
Okay seems like this is in good shape. @cmccabe have you tested this at all?
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user cmccabe commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44458395
So I was unable to reproduce the failure without this patch on CDH5.1.0.
It looks like it might have been a bad build (I didn't clean enough).
On a more positiv
Github user cmccabe commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44456990
I made a few small style changes suggested by Aaron Davidson.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as wel
Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44415223
+1 looks good.
I will note this again, the interfaces weren't cleaned up and tagged until
Hadoop 2.2. Hadoop 0.23 has been around for much longer then that and
Github user cmccabe commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44374458
I should clarify that when I say "fixes the trunk build" I mean it fixes
the trunk build when SPARK_YARN=1. When not building Spark-on-YARN at all, you
only need SPARK-18
Github user cmccabe commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44374139
So as I commented on the JIRA, using the public API here fixes
Spark-on-YARN for CDH5.1.0. That, plus the fact that this fixes the trunk
(hadoop 3.0.0) build, is probably
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-44371936
@tgravescs does this look okay to you?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project doe
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/850#discussion_r12930693
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -132,15 +134,19 @@ class Client(clientArgs: ClientArguments, hadoopConf:
Co
Github user cmccabe commented on a diff in the pull request:
https://github.com/apache/spark/pull/850#discussion_r12930683
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -132,15 +134,19 @@ class Client(clientArgs: ClientArguments, hadoopConf:
Github user cmccabe commented on a diff in the pull request:
https://github.com/apache/spark/pull/850#discussion_r12930649
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -132,15 +134,19 @@ class Client(clientArgs: ClientArguments, hadoopConf:
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/850#discussion_r12930178
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -132,15 +134,19 @@ class Client(clientArgs: ClientArguments, hadoopConf:
Co
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/850#discussion_r12930168
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -132,15 +134,19 @@ class Client(clientArgs: ClientArguments, hadoopConf:
Co
Github user sryza commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-43831445
+1. There's a separate version of Client under yarn/alpha that handles
0.23, so this change shouldn't cause any problems on that front.
---
If your project is set up for i
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-43831359
Does this API also work on Hadoop 0.23? We have some constraints in that we
want to support that.
---
If your project is set up for it, you can reply to this email and hav
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-43828253
+1 less private api usage yay!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/850#issuecomment-43824695
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your proj
GitHub user cmccabe opened a pull request:
https://github.com/apache/spark/pull/850
SPARK-1898: In deploy.yarn.Client, use YarnClient not YarnClientImpl
https://issues.apache.org/jira/browse/SPARK-1898
You can merge this pull request into a Git repository by running:
$ git pull
20 matches
Mail list logo
