Hi All, can I set spark.local.dir to HDFS location instead of /tmp folder ?
I tried setting up temp folder to HDFS but it didn't worked can spark.local.dir write to HDFS ? .set("spark.local.dir","hdfs://namednode/spark_tmp/") 16/07/05 15:35:47 ERROR DiskBlockManager: Failed to create local dir in hdfs://namenode/spark_tmp/. Ignoring this directory. java.io.IOException: Failed to create a temp directory (under hdfs://namenode/spark_tmp/) after 10 attempts! Thanks Sri -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-local-dir-to-HDFS-tp27291.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org