Repository: spark Updated Branches: refs/heads/master 01f183e84 -> 330bf5c99
[SPARK-20609][MLLIB][TEST] manually cleared 'spark.local.dir' before/after a test in ALSCleanerSuite ## What changes were proposed in this pull request? This PR is similar to #17869. Once` 'spark.local.dir'` is set. Unless this is manually cleared before/after a test. it could return the same directory even if this property is configured. and add before/after for each likewise in ALSCleanerSuite. ## How was this patch tested? existing test. Author: caoxuewen <cao.xue...@zte.com.cn> Closes #18537 from heary-cao/ALSCleanerSuite. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/330bf5c9 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/330bf5c9 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/330bf5c9 Branch: refs/heads/master Commit: 330bf5c99825afb6129577a34e6bed8b221a98cc Parents: 01f183e Author: caoxuewen <cao.xue...@zte.com.cn> Authored: Sat Jul 8 08:34:51 2017 +0100 Committer: Sean Owen <so...@cloudera.com> Committed: Sat Jul 8 08:34:51 2017 +0100 ---------------------------------------------------------------------- .../apache/spark/ml/recommendation/ALSSuite.scala | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/330bf5c9/mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala ---------------------------------------------------------------------- diff --git a/mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala b/mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala index b57fc8d..0a0fea2 100644 --- a/mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala +++ b/mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala @@ -29,6 +29,7 @@ import scala.language.existentials import com.github.fommil.netlib.BLAS.{getInstance => blas} import org.apache.commons.io.FileUtils import org.apache.commons.io.filefilter.TrueFileFilter +import org.scalatest.BeforeAndAfterEach import org.apache.spark._ import org.apache.spark.internal.Logging @@ -777,7 +778,20 @@ class ALSSuite } } -class ALSCleanerSuite extends SparkFunSuite { +class ALSCleanerSuite extends SparkFunSuite with BeforeAndAfterEach { + override def beforeEach(): Unit = { + super.beforeEach() + // Once `Utils.getOrCreateLocalRootDirs` is called, it is cached in `Utils.localRootDirs`. + // Unless this is manually cleared before and after a test, it returns the same directory + // set before even if 'spark.local.dir' is configured afterwards. + Utils.clearLocalRootDirs() + } + + override def afterEach(): Unit = { + Utils.clearLocalRootDirs() + super.afterEach() + } + test("ALS shuffle cleanup standalone") { val conf = new SparkConf() val localDir = Utils.createTempDir() --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org