[
https://issues.apache.org/jira/browse/SPARK-19592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Armin Braun updated SPARK-19592:
--------------------------------
Description:
This configuration for Surefire, Scalatest is duplicated in the parent POM as
well as the SBT build.
While this duplication cannot be removed in general it can at least be removed
for all system properties that simply result in a SparkConf setting I think.
Instead of having lines like
{code}
<spark.ui.enabled>false</spark.ui.enabled>
{code}
twice in the pom.xml
and once in SBT as
{code}
javaOptions in Test += "-Dspark.ui.enabled=false",
{code}
it would be a lot cleaner to simply have a
{code}
var conf: SparkConf
{code}
field in
{code}
org.apache.spark.SparkFunSuite
{code}
that has SparkConf set up with all the shared configuration that
`systemProperties` currently provide.
This makes the build a lot easier to maintain and makes tests more readable by
making the environment setup more explicit in the code.
(also it would allow running more tests straight from the IDE which is always a
nice thing imo)
was:
This configuration for Surefire, Scalatest is duplicated in the parent POM as
well as the SBT build.
While this duplication cannot be removed in general it can at least be removed
for all system properties that simply result in a SparkConf setting I think.
Instead of having lines like
{code}
<spark.ui.enabled>false</spark.ui.enabled>
{code}
twice in the pom.xml
and once in SBT as
{code}
javaOptions in Test += "-Dspark.ui.enabled=false",
{code}
it would be a lot cleaner to simply have a
{code}
var conf: SparkConf
{code}
field in
{code}
org.apache.spark.SparkFunSuite
{code}
that has SparkConf set up with all the shared configuration that
`systemProperties` currently provide.
This makes the build a lot easier to maintain and makes tests more readable by
making the environment setup more explicit in the code.
> Duplication in Test Configuration Relating to SparkConf Settings Should be
> Removed
> ----------------------------------------------------------------------------------
>
> Key: SPARK-19592
> URL: https://issues.apache.org/jira/browse/SPARK-19592
> Project: Spark
> Issue Type: Improvement
> Components: Tests
> Affects Versions: 2.1.0
> Environment: Applies to all Environments
> Reporter: Armin Braun
> Priority: Minor
>
> This configuration for Surefire, Scalatest is duplicated in the parent POM as
> well as the SBT build.
> While this duplication cannot be removed in general it can at least be
> removed for all system properties that simply result in a SparkConf setting I
> think.
> Instead of having lines like
> {code}
> <spark.ui.enabled>false</spark.ui.enabled>
> {code}
> twice in the pom.xml
> and once in SBT as
> {code}
> javaOptions in Test += "-Dspark.ui.enabled=false",
> {code}
> it would be a lot cleaner to simply have a
> {code}
> var conf: SparkConf
> {code}
> field in
> {code}
> org.apache.spark.SparkFunSuite
> {code}
> that has SparkConf set up with all the shared configuration that
> `systemProperties` currently provide.
> This makes the build a lot easier to maintain and makes tests more readable
> by making the environment setup more explicit in the code.
> (also it would allow running more tests straight from the IDE which is always
> a nice thing imo)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
