Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/17768#discussion_r113398866
--- Diff: core/src/test/scala/org/apache/spark/storage/LocalDirsSuite.scala
---
@@ -35,20 +35,31 @@ class LocalDirsSuite extends SparkFunSuite with
BeforeAndAfter {
test("Utils.getLocalDir() returns a valid directory, even if some local
dirs are missing") {
// Regression test for SPARK-2974
- assert(!new File("/NONEXISTENT_DIR").exists())
+ assert(!new File("/NONEXISTENT_PATH").exists())
val conf = new SparkConf(false)
.set("spark.local.dir",
s"/NONEXISTENT_PATH,${System.getProperty("java.io.tmpdir")}")
assert(new File(Utils.getLocalDir(conf)).exists())
}
test("SPARK_LOCAL_DIRS override also affects driver") {
// Regression test for SPARK-2975
- assert(!new File("/NONEXISTENT_DIR").exists())
+ assert(!new File("/NONEXISTENT_PATH").exists())
// spark.local.dir only contains invalid directories, but that's not a
problem since
// SPARK_LOCAL_DIRS will override it on both the driver and workers:
val conf = new SparkConfWithEnv(Map("SPARK_LOCAL_DIRS" ->
System.getProperty("java.io.tmpdir")))
.set("spark.local.dir", "/NONEXISTENT_PATH")
assert(new File(Utils.getLocalDir(conf)).exists())
}
+ test("Utils.getLocalDir() throws an exception if any temporary directory
cannot be retrieved") {
+ assert(!new File("/NONEXISTENT_PATH_ONE").exists())
+ assert(!new File("/NONEXISTENT_PATH_TWO").exists())
+ val conf = new SparkConf(false)
+ .set("spark.local.dir",
"/NONEXISTENT_PATH_ONE,/NONEXISTENT_PATH_TWO")
+ val message = intercept[IOException] {
+ Utils.getLocalDir(conf)
+ }.getMessage
+ assert(message ===
--- End diff --
I would only suggest not testing the exact message here, but it's not a big
deal. Maybe testing for the existence of the paths.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
