[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/11229


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187681228
  
Merged to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187589887
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187589891
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51736/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187589261
  
**[Test build #51736 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51736/consoleFull)**
 for PR 11229 at commit 
[`c85644e`](https://github.com/apache/spark/commit/c85644e69e0fc3e5d54a6e1758ab570b1dbb994c).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187583906
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51733/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187583905
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187583445
  
**[Test build #51733 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51733/consoleFull)**
 for PR 11229 at commit 
[`da42461`](https://github.com/apache/spark/commit/da4246114c2e304a2ea3706a832798eaf132037e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187548150
  
**[Test build #51736 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51736/consoleFull)**
 for PR 11229 at commit 
[`c85644e`](https://github.com/apache/spark/commit/c85644e69e0fc3e5d54a6e1758ab570b1dbb994c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187543399
  
**[Test build #51733 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51733/consoleFull)**
 for PR 11229 at commit 
[`da42461`](https://github.com/apache/spark/commit/da4246114c2e304a2ea3706a832798eaf132037e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

Github user jerryshao commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187538003
  
OK, sure.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-187323397
  
LGTM, unfortunately this no longer merges. Can you resolve the conflicts?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186993223
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186993224
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51650/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186992965
  
**[Test build #51650 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51650/consoleFull)**
 for PR 11229 at commit 
[`330b0c4`](https://github.com/apache/spark/commit/330b0c44f5b42bb5615b4dcd37d86507b7ea8b6e).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186967084
  
**[Test build #51650 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51650/consoleFull)**
 for PR 11229 at commit 
[`330b0c4`](https://github.com/apache/spark/commit/330b0c44f5b42bb5615b4dcd37d86507b7ea8b6e).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

Github user jerryshao commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186961739
  
Thanks @andrewor14 for your comments, I've updated the codes accordingly, 
please review.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186349698
  
This looks great. I left a few minor comments. Once you address them I'll 
go ahead and merge this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53497343
  
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -237,6 +237,7 @@ class SparkContext(config: SparkConf) extends Logging 
with ExecutorAllocationCli
   def jars: Seq[String] = _jars
   def files: Seq[String] = _files
   def master: String = _conf.get("spark.master")
+  def deployMode: String = 
_conf.getOption("spark.submit.deployMode").getOrElse("client")
--- End diff --

I'd just like to point out that this is a public API that we won't be able 
to remove easily in the future. This is probably OK since I don't see the 
concept of deploy mode going away ever, but this is worth a mention.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53497148
  
--- Diff: project/MimaExcludes.scala ---
@@ -248,6 +248,9 @@ object MimaExcludes {
 
ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.graphx.Graph.mapReduceTriplets"),
 
ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.graphx.Graph.mapReduceTriplets$default$3"),
 
ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.graphx.impl.GraphImpl.mapReduceTriplets")
+  ) ++ Seq(
+// SPARK-13220 Deprecate yarn-client and yarn-cluster mode
+
ProblemFilters.exclude[MissingMethodProblem]("org.apache.spark.SparkContext.org$apache$spark$SparkContext$$createTaskScheduler")
--- End diff --

can you put this on the next line


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53496981
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -244,19 +248,16 @@ object SparkSubmit {
 // and deploy mode, we have some logic to infer the master and deploy 
mode
--- End diff --

this comment is outdated


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53496884
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -226,7 +226,11 @@ object SparkSubmit {
 
 // Set the cluster manager
 val clusterManager: Int = args.master match {
-  case m if m.startsWith("yarn") => YARN
+  case "yarn" => YARN
+  case "yarn-client" | "yarn-cluster" =>
--- End diff --

if I specify `yarn-anything` now this will fail with an error message that 
is not helpful:
`Master must start with yarn, spark, mesos, or local` (my master *does* 
start with yarn). We should handle that case or change the error message.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53496924
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -226,7 +226,11 @@ object SparkSubmit {
 
 // Set the cluster manager
 val clusterManager: Int = args.master match {
-  case m if m.startsWith("yarn") => YARN
+  case "yarn" => YARN
+  case "yarn-client" | "yarn-cluster" =>
--- End diff --

maybe something like `Master must either be yarn or start with spark, mesos 
or local`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53496574
  
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -1595,10 +1594,8 @@ class SparkContext(config: SparkConf) extends 
Logging with ExecutorAllocationCli
 key = uri.getScheme match {
   // A JAR file which exists only on the driver node
   case null | "file" =>
-// yarn-standalone is deprecated, but still supported
-if (SparkHadoopUtil.get.isYarnMode() &&
-(master == "yarn-standalone" || master == "yarn-cluster")) 
{
-  // In order for this to work in yarn-cluster mode the user 
must specify the
+if (SparkHadoopUtil.get.isYarnMode() && (master == "yarn" && 
deployMode == "cluster")) {
--- End diff --

not your code, but this is kind of weird. At this point of master is `yarn` 
we should be pretty sure `isYarnMode` is true.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[andrewor14]

Github user andrewor14 commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53495418
  
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -503,6 +503,23 @@ class SparkConf(loadDefaults: Boolean) extends 
Cloneable with Logging {
 set("spark.executor.instances", value)
   }
 }
+
+if (contains("spark.master") && 
get("spark.master").startsWith("yarn-")) {
+  val warning = s"spark.master ${get("spark.master")} is deprecated in 
Spark 2.0+, please " +
+"instead use \"yarn\" with specified deploy mode."
+
+  get("spark.master") match {
+case "yarn-cluster" =>
+  logWarning(warning)
+  set("spark.master", "yarn")
+  set("spark.submit.deployMode", "cluster")
--- End diff --

should we also verify separately that `spark.submit.deployMode` must either 
be cluster or client?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186116817
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51524/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186116815
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186116594
  
**[Test build #51524 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51524/consoleFull)**
 for PR 11229 at commit 
[`480298c`](https://github.com/apache/spark/commit/480298cbe277dacf7901fb81019037fe74f13332).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-186082895
  
**[Test build #51524 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51524/consoleFull)**
 for PR 11229 at commit 
[`480298c`](https://github.com/apache/spark/commit/480298cbe277dacf7901fb81019037fe74f13332).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

Github user jerryshao commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53424443
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -245,18 +249,20 @@ object SparkSubmit {
 // from each other if only one is specified, or exit early if they are 
at odds.
 if (clusterManager == YARN) {
   if (args.master == "yarn-standalone") {
-printWarning("\"yarn-standalone\" is deprecated. Use 
\"yarn-cluster\" instead.")
+printWarning("\"yarn-standalone\" is deprecated. Use \"yarn\" " +
--- End diff --

I'm planning to remove the support of "yarn-standalone". Like what @srowen 
said, I don't there's many user still use "yarn-standalone" for now.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[vanzin]

Github user vanzin commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185884759
  
LGTM aside from a couple of minor things, and the issue Sean pointed out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[vanzin]

Github user vanzin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53369752
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -245,18 +249,20 @@ object SparkSubmit {
 // from each other if only one is specified, or exit early if they are 
at odds.
 if (clusterManager == YARN) {
   if (args.master == "yarn-standalone") {
-printWarning("\"yarn-standalone\" is deprecated. Use 
\"yarn-cluster\" instead.")
+printWarning("\"yarn-standalone\" is deprecated. Use \"yarn\" " +
--- End diff --

So, if someone says "--master yarn-standalone", it seems like they'll see 3 
different warnings in different places, right?
- one in L231 above
- one here
- one in the driver's output (SparkContext also logs something)

This particular one seems a little too redundant now, probably should be 
removed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[vanzin]

Github user vanzin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53369673
  
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -226,7 +226,11 @@ object SparkSubmit {
 
 // Set the cluster manager
 val clusterManager: Int = args.master match {
-  case m if m.startsWith("yarn") => YARN
+  case "yarn" => YARN
+  case "yarn-client" | "yarn-cluster" | "yarn-standalone" =>
+printWarning(s"Master ${args.master} is deprecated since 2.0." +
+  " Please use master \"yarn\" with specified deploy mode instead")
--- End diff --

nit: missing period.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[vanzin]

Github user vanzin commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53368347
  
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -503,6 +503,23 @@ class SparkConf(loadDefaults: Boolean) extends 
Cloneable with Logging {
 set("spark.executor.instances", value)
   }
 }
+
+if (contains("spark.master") && 
get("spark.master").startsWith("yarn")) {
+  val warning = s"spark.master ${get("spark.master")} is deprecated in 
Spark 2.0+, please " +
+"instead use \"yarn\" with specified deploy mode."
+
+  get("spark.master") match {
+case "yarn-cluster" | "yarn-standalone" =>
+  logWarning(warning)
+  set("spark.master", "yarn")
+  set("spark.submit.deployMode", "cluster")
+case "yarn-client" =>
+  logWarning(warning)
+  set("spark.master", "yarn")
+  set("spark.submit.deployMode", "client")
+case _ => Unit
--- End diff --

nit: `Unit` is not necessary here.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185650892
  
That looks solid @jerryshao . CC @vanzin . It looks like you've searched 
for and replaced any actual use of "yarn-client" and "yarn-cluster" internally, 
and have only kept support for these strings in the scripts? that's great. If 
anyone then feels strongly about removing that support entirely, it's a simple 
change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53297383
  
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -380,10 +381,8 @@ class SparkContext(config: SparkConf) extends Logging 
with ExecutorAllocationCli
 }
 
 // System property spark.yarn.app.id must be set if user code ran by 
AM on a YARN cluster
-// yarn-standalone is deprecated, but still supported
-if ((master == "yarn-cluster" || master == "yarn-standalone") &&
--- End diff --

FWIW I don't think `yarn-standalone` has existed for a long time and can be 
ignored, yes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/11229#discussion_r53297311
  
--- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
@@ -503,6 +503,23 @@ class SparkConf(loadDefaults: Boolean) extends 
Cloneable with Logging {
 set("spark.executor.instances", value)
   }
 }
+
+if (contains("spark.master") && 
get("spark.master").startsWith("yarn")) {
--- End diff --

Don't you need to check `startsWith("yarn-")`? because "yarn" starts with 
"yarn".


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-18098
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51465/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-18096
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185554977
  
**[Test build #51465 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51465/consoleFull)**
 for PR 11229 at commit 
[`14b08f6`](https://github.com/apache/spark/commit/14b08f6e99408eaf1307d5190979b768ad2b77fa).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185531467
  
**[Test build #51465 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51465/consoleFull)**
 for PR 11229 at commit 
[`14b08f6`](https://github.com/apache/spark/commit/14b08f6e99408eaf1307d5190979b768ad2b77fa).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

Github user jerryshao commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185529327
  
Jenkins, retest this please.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185526103
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51461/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185526101
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185525516
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51460/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185525515
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

Github user jerryshao commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185525256
  
@srowen , I just change back to only deprecate these two modes, user could 
still use them with warning log. Please review, thanks a lot.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185172746
  
I suppose I have a mild preference for retaining them in 2.x just because 
they're so widely used, and deprecating them -- certainly it doesn't seem like 
a lot of burden to do so. However I don't feel strongly about it. If someone 
feels fairly strongly that they should be removed I'm OK with that


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

Github user jerryshao commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185165800
  
So your meaning is that we still need to keep these two modes just 
deprecate them? 

What is your suggestion @andrewor14 ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[srowen]

Github user srowen commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185161132
  
This actually removes support, right? I think the idea was to just 
deprecate it, even for 2.x


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185159189
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185159190
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51423/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185159008
  
**[Test build #51423 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51423/consoleFull)**
 for PR 11229 at commit 
[`bf1faed`](https://github.com/apache/spark/commit/bf1faeda09eaa2fefac1807adb6784b3fd5b8c48).
 * This patch **fails PySpark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185114117
  
**[Test build #51423 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51423/consoleFull)**
 for PR 11229 at commit 
[`bf1faed`](https://github.com/apache/spark/commit/bf1faeda09eaa2fefac1807adb6784b3fd5b8c48).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185084845
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185084851
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51409/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185084564
  
**[Test build #51409 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51409/consoleFull)**
 for PR 11229 at commit 
[`bfcd14c`](https://github.com/apache/spark/commit/bfcd14cb68ebb21ca3a5a1a6667758c76c77178d).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185032333
  
**[Test build #51409 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51409/consoleFull)**
 for PR 11229 at commit 
[`bfcd14c`](https://github.com/apache/spark/commit/bfcd14cb68ebb21ca3a5a1a6667758c76c77178d).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185030818
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51407/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185030814
  
Build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185011892
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[AmplabJenkins]

Github user AmplabJenkins commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185011896
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/51402/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185011817
  
**[Test build #51402 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51402/consoleFull)**
 for PR 11229 at commit 
[`acfba66`](https://github.com/apache/spark/commit/acfba66071f95c11847535995c4ce574b345549c).
 * This patch **fails MiMa tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[SparkQA]

Github user SparkQA commented on the pull request:

https://github.com/apache/spark/pull/11229#issuecomment-185006361
  
**[Test build #51402 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/51402/consoleFull)**
 for PR 11229 at commit 
[`acfba66`](https://github.com/apache/spark/commit/acfba66071f95c11847535995c4ce574b345549c).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request: [SPARK-13220][Core]deprecate yarn-client and y...

[jerryshao]

GitHub user jerryshao opened a pull request:

https://github.com/apache/spark/pull/11229

[SPARK-13220][Core]deprecate yarn-client and yarn-cluster mode



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jerryshao/apache-spark SPARK-13220

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/11229.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #11229


commit acfba66071f95c11847535995c4ce574b345549c
Author: jerryshao 
Date:   2016-02-17T03:43:15Z

deprecate yarn-client and yarn-cluster mode




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org