[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/4522 [SPARK-5732][CORE]:Add an option to print the spark version in spark script. Naturally, we need to add an option to print the spark version in spark script. It is pretty common in script tool. There are some little changes in `verbose`. Its abbreviation, i.e. "-v", is removed and used by option "--version". Maybe, it will influence the established habits. Any ideas? You can merge this pull request into a Git repository by running: $ git pull https://github.com/uncleGen/spark master-clean-150211 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4522.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4522 commit 463f02c1f5e4c7b0a6f3cd12a817472c6b19bce0 Author: uncleGen Date: 2015-02-11T05:00:18Z minor changes commit 015ddeed95df5dc35be4ed179c26186a9713a49b Author: uncleGen Date: 2015-02-11T05:01:50Z minor changes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-73835711 [Test build #27274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27274/consoleFull) for PR 4522 at commit [`015ddee`](https://github.com/apache/spark/commit/015ddeed95df5dc35be4ed179c26186a9713a49b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-73838664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27274/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-73838659 [Test build #27274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27274/consoleFull) for PR 4522 at commit [`015ddee`](https://github.com/apache/spark/commit/015ddeed95df5dc35be4ed179c26186a9713a49b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user uncleGen commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-73839883 An irrelevant test failure in `DirectKafkaStreamSuite` introduced by [PR](4384) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24478679
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args:
Seq[String], env: Map[String, St
case ("--help" | "-h") :: tail =>
printUsageAndExit(0)
- case ("--verbose" | "-v") :: tail =>
+ case ("--verbose") :: tail =>
verbose = true
parse(tail)
+ case ("--version" | "-v") :: tail =>
--- End diff --
Eh, I don't think you should just change behavior of `-v`!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24478804
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args:
Seq[String], env: Map[String, St
case ("--help" | "-h") :: tail =>
printUsageAndExit(0)
- case ("--verbose" | "-v") :: tail =>
+ case ("--verbose") :: tail =>
verbose = true
parse(tail)
+ case ("--version" | "-v") :: tail =>
+SparkSubmit.printVersionAndExit()
--- End diff --
Some binaries, like Java, don't exit just because the version is printed.
Rather than push some flow control and exit from the argument parsing code,
this could become another flag in the args object, that later causes version to
print in the normal flow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24490981
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args:
Seq[String], env: Map[String, St
case ("--help" | "-h") :: tail =>
printUsageAndExit(0)
- case ("--verbose" | "-v") :: tail =>
+ case ("--verbose") :: tail =>
verbose = true
parse(tail)
+ case ("--version" | "-v") :: tail =>
--- End diff --
Yes, it remains open to question.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24491713
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args:
Seq[String], env: Map[String, St
case ("--help" | "-h") :: tail =>
printUsageAndExit(0)
- case ("--verbose" | "-v") :: tail =>
+ case ("--verbose") :: tail =>
verbose = true
parse(tail)
+ case ("--version" | "-v") :: tail =>
+SparkSubmit.printVersionAndExit()
--- End diff --
IMHO, we may just need to print the version information as simple as
possible, just like what the option `help` dose.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24492501
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args:
Seq[String], env: Map[String, St
case ("--help" | "-h") :: tail =>
printUsageAndExit(0)
- case ("--verbose" | "-v") :: tail =>
+ case ("--verbose") :: tail =>
verbose = true
parse(tail)
+ case ("--version" | "-v") :: tail =>
--- End diff --
IMHO it would be better to keep the compatible behavior to the old version,
not to change `-v`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user uncleGen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24495488
--- Diff:
core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala ---
@@ -413,10 +413,13 @@ private[spark] class SparkSubmitArguments(args:
Seq[String], env: Map[String, St
case ("--help" | "-h") :: tail =>
printUsageAndExit(0)
- case ("--verbose" | "-v") :: tail =>
+ case ("--verbose") :: tail =>
verbose = true
parse(tail)
+ case ("--version" | "-v") :: tail =>
--- End diff --
OK, I will remain the behavior of `-v`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-74172115 Yeah we cannot change the behavior of `-v`, though we could also print the version in the usage message. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-74173008 LGTM if the behavior of `-v` is reverted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-74212015 [Test build #27420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27420/consoleFull) for PR 4522 at commit [`9f2127c`](https://github.com/apache/spark/commit/9f2127cd0863437cf1f5add51d0709165b7f3fe8). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-74217797 [Test build #27420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27420/consoleFull) for PR 4522 at commit [`9f2127c`](https://github.com/apache/spark/commit/9f2127cd0863437cf1f5add51d0709165b7f3fe8). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-74217801 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27420/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24680879
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -89,6 +90,17 @@ object SparkSubmit {
printStream.println("Run with --help for usage help or --verbose for
debug output")
exitFn()
}
+ private[spark] def printVersionAndExit() = {
--- End diff --
need new line before this and `Unit` return type
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/4522#discussion_r24680952
--- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala ---
@@ -40,6 +40,7 @@ import org.apache.ivy.plugins.resolver.{ChainResolver,
IBiblioResolver}
import org.apache.spark.deploy.rest._
import org.apache.spark.executor._
import org.apache.spark.util.{ChildFirstURLClassLoader,
MutableURLClassLoader, Utils}
+import org.apache.spark._
--- End diff --
I would just explicitly import `SPARK_VERSION` since that's the only thing
we need here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4522#issuecomment-74294528 Merged into master and 1.3 after fixing the comments myself. I realize that the neighboring functions also do not have the correct return type so I added those too. Thanks @uncleGen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-5732][CORE]:Add an option to print the ...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4522 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
