[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-48248121 @codeboyyong The PR was not closed automatically. Could you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user codeboyyong closed the pull request at: https://github.com/apache/spark/pull/1099 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47968583 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47968908 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47968895 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47978598 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47978602 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16310/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47982406 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47982776 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47982767 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47990763 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16312/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47990760 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47991050 Merged. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190528 No, just want to see Jenkins happy. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190546 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190746 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47190739 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47191952 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47191953 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16138/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47246091 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47246313 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47246327 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47250103 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16161/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47250101 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47250277 Seems that the jenkins is missing the python runtime. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user codeboyyong commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47260625 Would this be caused by this commit? Should we revert it and try again ? John. On Jun 26, 2014, at 9:50 AM, DB Tsai notificati...@github.com wrote: Seems that the jenkins is missing the python runtime. â Reply to this email directly or view it on GitHub. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47291043 I think `PYSPARK_PYTHON` is set to `/usr/local/bin/python2.7` in Jenkins but it doesn't exist. @pwendell ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-47125253 @mengxr any further comments on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46814540 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46816642 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46816645 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16019/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46924196 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46924244 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46924255 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46925751 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46925752 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16045/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46271992 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46272002 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46273682 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46273683 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15845/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user codeboyyong commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46276348 @mengxr . I fix the format . And I did the test on hadoop-2.0.5-alpha and hadoop 2.2.0. The command line now works as usual. And my local yarn cluster is not working .I might test the run time exit later. Will let you know the result --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46375206 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46375197 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46377483 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46377486 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15857/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
GitHub user codeboyyong opened a pull request: https://github.com/apache/spark/pull/1099 [SPARK-1516]Throw exception in yarn client instead of run system.exit [SPARK-1516]Throw exception in yarn client instead of run system.exit directly. All the changes is in the package of org.apache.spark.deploy.yarn: 1) Throw IllegalArgumentException in ClinetArguments instead of exit directly. 2) In Client's main method, if exception is caught, it will exit with code 1, otherwise exit with code 0. 3) In YarnClientSchedulerBackend's start method, if IllegalArgumentException is caught, it will exit with code 1, otherwise throw that exception. 4) Fix some message typo in the Client.scala After the fix, if user integrate the spark yarn client into their applications, when the argument is wrong or the running is finished, the application won't be terminated. You can merge this pull request into a Git repository by running: $ git pull https://github.com/codeboyyong/spark branch-0.9 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/1099.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1099 commit 6f71cd85916edd97af0afca13f300763bcc6c737 Author: John Zhao codeboyy...@gmail.com Date: 2014-06-16T22:55:49Z [SPARK-1516]Throw exception in yarn client instead of run system.exit directly. All the changes is in the package of org.apache.spark.deploy.yarn: 1) Throw IllegalArgumentException in ClinetArguments instead of exit directly. 2) In Client's main method, if exception is caught, it will exit with code 1, otherwise exit with code 0. 3) In YarnClientSchedulerBackend's start method, if IllegalArgumentException is caught, it will exit with code 1, otherwise throw that exception. 4) Fix some message typo in the Client.scala After the fix, if user integrate the spark yarn client into their applications, when the argument is wrong or the running is finished, the application won't be terminated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46251622 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46251624 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15827/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/1099#discussion_r13843020
--- Diff:
yarn/common/src/main/scala/org/apache/spark/scheduler/cluster/YarnClientSchedulerBackend.scala
---
@@ -73,10 +73,18 @@ private[spark] class YarnClientSchedulerBackend(
.foreach { case (optName, optParam) = addArg(optName, optParam,
argsArrayBuf) }
logDebug(ClientArguments called with: + argsArrayBuf)
-val args = new ClientArguments(argsArrayBuf.toArray, conf)
-client = new Client(args, conf)
-appId = client.runApp()
-waitForApp()
+try {
+ val args = new ClientArguments(argsArrayBuf.toArray, conf)
+ client = new Client(args, conf)
+ appId = client.runApp()
+ waitForApp()
+} catch {
+ case e: IllegalArgumentException = {
+Console.err.println(e.getMessage)
+System.exit(1)
+ }
+ case x:Throwable = throw x
--- End diff --
Remove this line since this is the default behavior.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/1099#discussion_r13843025
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -130,7 +129,8 @@ class Client(args: ClientArguments, conf:
Configuration, sparkConf: SparkConf)
).foreach { case(cond, errStr) =
if (cond) {
logError(errStr)
-args.printUsageAndExit(1)
+throw new IllegalArgumentException(args.getUsageMessage())
+
--- End diff --
Remove empty line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/1099#discussion_r13843041
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -160,15 +160,19 @@ class Client(args: ClientArguments, conf:
Configuration, sparkConf: SparkConf)
// If we have requested more then the clusters max for a single
resource then exit.
if (args.workerMemory maxMem) {
- logError(Required worker memory (%d MB), is above the max threshold
(%d MB) of this cluster..
-format(args.workerMemory, maxMem))
- System.exit(1)
+ val errorMessage = Required worker memory (%d MB), is above the max
threshold (%d MB) of this cluster.
--- End diff --
Too wide.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/1099#discussion_r13843048
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -160,15 +160,19 @@ class Client(args: ClientArguments, conf:
Configuration, sparkConf: SparkConf)
// If we have requested more then the clusters max for a single
resource then exit.
if (args.workerMemory maxMem) {
- logError(Required worker memory (%d MB), is above the max threshold
(%d MB) of this cluster..
-format(args.workerMemory, maxMem))
- System.exit(1)
+ val errorMessage = Required worker memory (%d MB), is above the max
threshold (%d MB) of this cluster.
+.format(args.workerMemory, maxMem)
+ logError(errorMessage)
+ throw new IllegalArgumentException(errorMessage)
+
}
val amMem = args.amMemory + YarnAllocationHandler.MEMORY_OVERHEAD
if (amMem maxMem) {
- logError(Required AM memory (%d) is above the max threshold (%d) of
this cluster.
-format(args.amMemory, maxMem))
- System.exit(1)
+ val errorMessage = Required AM memory (%d) is above the max
threshold (%d) of this cluster
+.format(args.amMemory, maxMem)
+ logError(errorMessage)
+ throw new IllegalArgumentException(errorMessage)
+
--- End diff --
same.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/1099#discussion_r13843046
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -160,15 +160,19 @@ class Client(args: ClientArguments, conf:
Configuration, sparkConf: SparkConf)
// If we have requested more then the clusters max for a single
resource then exit.
if (args.workerMemory maxMem) {
- logError(Required worker memory (%d MB), is above the max threshold
(%d MB) of this cluster..
-format(args.workerMemory, maxMem))
- System.exit(1)
+ val errorMessage = Required worker memory (%d MB), is above the max
threshold (%d MB) of this cluster.
+.format(args.workerMemory, maxMem)
+ logError(errorMessage)
+ throw new IllegalArgumentException(errorMessage)
+
--- End diff --
Remove empty line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/1099#issuecomment-46267457 @codeboyyong Thanks for submitting the patch! It looks good to me except a few style issues. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45835283 @mengxr Do you think it's in good shape now? This is the only issue blocking us using vanilla spark. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13689598
--- Diff:
yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -179,8 +178,17 @@ object Client {
System.setProperty(SPARK_YARN_MODE, true)
val sparkConf = new SparkConf
-val args = new ClientArguments(argStrings, sparkConf)
-new Client(args, sparkConf).run
+try{
--- End diff --
`try {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13690774
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
---
@@ -18,6 +18,7 @@
package org.apache.spark.deploy.yarn
import scala.collection.mutable.{ArrayBuffer, HashMap}
+import scala.IllegalArgumentException
--- End diff --
Is it necessary? Scala imports `java.lang.IAE` by default.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13690778
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -24,6 +24,7 @@ import java.nio.ByteBuffer
import scala.collection.JavaConversions._
import scala.collection.mutable.{HashMap, ListBuffer, Map}
import scala.util.{Try, Success, Failure}
+import scala.IllegalArgumentException
--- End diff --
Not necessary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13690782
--- Diff:
yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala ---
@@ -188,9 +187,18 @@ object Client {
// see Client#setupLaunchEnv().
System.setProperty(SPARK_YARN_MODE, true)
val sparkConf = new SparkConf()
-val args = new ClientArguments(argStrings, sparkConf)
-new Client(args, sparkConf).run()
+try{
--- End diff --
`try {`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13690775
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
---
@@ -138,11 +139,10 @@ class ClientArguments(val args: Array[String], val
sparkConf: SparkConf) {
}
- def printUsageAndExit(exitCode: Int, unknownParam: Any = null) {
-if (unknownParam != null) {
- System.err.println(Unknown/unsupported param + unknownParam)
-}
-System.err.println(
+ def getUsageMessage(unknownParam: Any = null): String = {
+val message = if (unknownParam != null) Unknown/unsupported param +
unknownParam+ \n else
--- End diff --
`sUnknown/unsupported param $unknownParam\n` looks better.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user codeboyyong commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45928754 @mengxr, I made the change based on your comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45965012 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45965021 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45965031 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45965023 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45965295 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45966210 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45969012 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15744/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45969007 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45969010 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15745/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45969006 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/490 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45976550 @codeboyyong I've merged this. Could you please make a patch for branch-0.9? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user codeboyyong commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45976588 Sure, will do this at weekend. On Jun 12, 2014, at 10:13 PM, Xiangrui Meng notificati...@github.com wrote: @codeboyyong I've merged this. Could you please make a patch for branch-0.9? Thanks! â Reply to this email directly or view it on GitHub. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45578114 @codeboyyong It is not mergable now. Do you mind merging the master branch and also create a separate PR for branch-0.9? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user codeboyyong commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45681667 I merged it to the master now. Will do 0.9 soon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13624385
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -95,15 +96,18 @@ trait ClientBase extends Logging {
// If we have requested more then the clusters max for a single
resource then exit.
if (args.executorMemory maxMem) {
- logError(Required executor memory (%d MB), is above the max
threshold (%d MB) of this cluster..
-format(args.executorMemory, maxMem))
- System.exit(1)
+ val errorMessage =
+Required executor memory (%d MB), is above the max threshold (%d
MB) of this cluster..
+format(args.executorMemory, maxMem)
+ logError(errorMessage)
+ throw new IllegalArgumentException(errorMessage)
}
val amMem = args.amMemory + YarnAllocationHandler.MEMORY_OVERHEAD
if (amMem maxMem) {
- logError(Required AM memory (%d) is above the max threshold (%d) of
this cluster.
-format(args.amMemory, maxMem))
- System.exit(1)
+ val errorMessage =Required AM memory (%d) is above the max
threshold (%d) of this cluster.
--- End diff --
Please add a space after =
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13624580
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -95,15 +96,18 @@ trait ClientBase extends Logging {
// If we have requested more then the clusters max for a single
resource then exit.
if (args.executorMemory maxMem) {
- logError(Required executor memory (%d MB), is above the max
threshold (%d MB) of this cluster..
-format(args.executorMemory, maxMem))
- System.exit(1)
+ val errorMessage =
+Required executor memory (%d MB), is above the max threshold (%d
MB) of this cluster..
+format(args.executorMemory, maxMem)
--- End diff --
Move the . to the new line
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r13624615
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -95,15 +96,18 @@ trait ClientBase extends Logging {
// If we have requested more then the clusters max for a single
resource then exit.
if (args.executorMemory maxMem) {
- logError(Required executor memory (%d MB), is above the max
threshold (%d MB) of this cluster..
-format(args.executorMemory, maxMem))
- System.exit(1)
+ val errorMessage =
+Required executor memory (%d MB), is above the max threshold (%d
MB) of this cluster..
+format(args.executorMemory, maxMem)
+ logError(errorMessage)
+ throw new IllegalArgumentException(errorMessage)
}
val amMem = args.amMemory + YarnAllocationHandler.MEMORY_OVERHEAD
if (amMem maxMem) {
- logError(Required AM memory (%d) is above the max threshold (%d) of
this cluster.
-format(args.amMemory, maxMem))
- System.exit(1)
+ val errorMessage =Required AM memory (%d) is above the max
threshold (%d) of this cluster.
--- End diff --
move the . to the newline
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-45274129 I think this change would be good too. I think we should also look into what our exit code is and how it would fix into workflow managers like oozie. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41822987 This change makes sense to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
GitHub user codeboyyong opened a pull request: https://github.com/apache/spark/pull/490 [SPARK-1516]Throw exception in yarn client instead of run system.exit directly. All the changes is in the package of org.apache.spark.deploy.yarn: 1) Throw exception in ClinetArguments and ClientBase instead of exit directly. 2) in Client's main method, if exception is caught, it will exit with code 1, otherwise exit with code 0. After the fix, if user integrate the spark yarn client into their applications, when the argument is wrong or the running is finished, the application won't be terminated. You can merge this pull request into a Git repository by running: $ git pull https://github.com/AlpineNow/spark jira_1516_systemexit_inyarnclient Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/490.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #490 commit f7a037988a9477b2a4e7b8fce0720e5ae3eda0ac Author: John Zhao jz...@alpinenow.com Date: 2014-04-22T22:58:35Z [SPARK-1516]Throw exception in yarn clinet instead of run system.exit directly. All the changes is in the package of org.apache.spark.deploy.yarn: 1) Add a ClientException with an exitCode 2) Throws exception in ClinetArguments and ClientBase instead of exit directly 3) in Client's main method, catch exception and exit with the exitCode. After the fix, if user integrate the spark yarn cline into their applications, when the argument is wrong or the running is finished, the application will not exit. Now the exit only happens in command line running. commit 5ae631d688d25a2f1ca79319c490357a3d4caad4 Author: John Zhao jz...@alpinenow.com Date: 2014-04-23T00:45:19Z Code formatting --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41113724 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on a diff in the pull request:
https://github.com/apache/spark/pull/490#discussion_r11883381
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala ---
@@ -77,7 +78,8 @@ trait ClientBase extends Logging {
).foreach { case(cond, errStr) =
if (cond) {
logError(errStr)
-args.printUsageAndExit(1)
+throw new IllegalArgumentException(args.getUsageMessage())
+
--- End diff --
Remove this empty line.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41114289 Jenkins, add to whitelist. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41115145 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41115323 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41115328 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: [SPARK-1516]Throw exception in yarn client ins...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/490#issuecomment-41117046 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14350/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
