[
https://issues.apache.org/jira/browse/SPARK-4893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen updated SPARK-4893:
------------------------------
Description:
Several of our tests call {{System.setProperty}} (or test code which implicitly
sets system properties) and don't always reset/clear the modified properties,
which can create ordering dependencies between tests and cause
[hard-to-diagnose
failures|https://github.com/apache/spark/pull/3561#issuecomment-67583605].
I think we can remove most of these System.setProperty calls and replace them
by proper usage of SparkConf.
For the remaining cases where we still need to use System.setProperty, I think
that we should use a ScalaTest fixture / mixin to automatically snapshot and
restore system properties after test runs.
was:
Several of our tests call {{System.setProperty}} (or test code which implicitly
sets system properties) and don't always reset/clear the modified properties,
which can create ordering dependencies between tests and cause
[hard-to-diagnose
failures|https://github.com/apache/spark/pull/3561#issuecomment-67583605].
I think that we should use a ScalaTest fixture / mixin to automatically
snapshot and restore system properties after test runs.
> Clean up uses of System.setProperty in unit tests
> -------------------------------------------------
>
> Key: SPARK-4893
> URL: https://issues.apache.org/jira/browse/SPARK-4893
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Josh Rosen
> Assignee: Josh Rosen
>
> Several of our tests call {{System.setProperty}} (or test code which
> implicitly sets system properties) and don't always reset/clear the modified
> properties, which can create ordering dependencies between tests and cause
> [hard-to-diagnose
> failures|https://github.com/apache/spark/pull/3561#issuecomment-67583605].
> I think we can remove most of these System.setProperty calls and replace them
> by proper usage of SparkConf.
> For the remaining cases where we still need to use System.setProperty, I
> think that we should use a ScalaTest fixture / mixin to automatically
> snapshot and restore system properties after test runs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
