[
https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-14279:
----------------------------------
Description:
Right now the spark-submit --version and other parts of the code pick up
version information from a static SPARK_VERSION. We would want to pick the
version from the pom.version probably stored inside a properties file. Also, it
might be nice to have other details like branch, build information and other
specific details when having a spark-submit --version
the motivation is to more easily tie this to automated continuous integration
and deployment and to easily have traceability.
Part of this is right now you have to manually change a java file to change the
version that comes out when you run spark-submit --version. With continuous
integration the build numbers could be something like 1.6.1.X (where X
increments on each change) and I want to see the exact version easily. Having
to manually change a java file makes that hard. obviously that should make the
apache spark releases easier as you don't have to manually change this file as
well.
The other important part for me is the git information. This easily lets me
trace it back to exact commits. We have a multi-tenant YARN cluster and users
can run many different versions at once. I want to be able to see exactly which
version they are running. The reason to know exact version can range from
helping debug some problem to making sure someone didn't hack something in
Spark to cause bad things (generally they should use approved version), etc.
was:Right now the spark-submit --version and other parts of the code pick up
version information from a static SPARK_VERSION. We would want to pick the
version from the pom.version probably stored inside a properties file. Also, it
might be nice to have other details like branch, build information and other
specific details when having a spark-submit --version
> Improve the spark build to pick the version information from the pom file
> instead of package.scala
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-14279
> URL: https://issues.apache.org/jira/browse/SPARK-14279
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Reporter: Sanket Reddy
> Assignee: Sanket Reddy
> Priority: Minor
>
> Right now the spark-submit --version and other parts of the code pick up
> version information from a static SPARK_VERSION. We would want to pick the
> version from the pom.version probably stored inside a properties file. Also,
> it might be nice to have other details like branch, build information and
> other specific details when having a spark-submit --version
> the motivation is to more easily tie this to automated continuous integration
> and deployment and to easily have traceability.
> Part of this is right now you have to manually change a java file to change
> the version that comes out when you run spark-submit --version. With
> continuous integration the build numbers could be something like 1.6.1.X
> (where X increments on each change) and I want to see the exact version
> easily. Having to manually change a java file makes that hard. obviously that
> should make the apache spark releases easier as you don't have to manually
> change this file as well.
> The other important part for me is the git information. This easily lets me
> trace it back to exact commits. We have a multi-tenant YARN cluster and users
> can run many different versions at once. I want to be able to see exactly
> which version they are running. The reason to know exact version can range
> from helping debug some problem to making sure someone didn't hack something
> in Spark to cause bad things (generally they should use approved version),
> etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
