Hello, I notice that when I run spark-submit, a temporary directory containing all the jars and resource files is created under /tmp (for example, /tmp/spark-fd1b77fc-50f4-4b1c-a122-5cf36969407c).
Sometimes this directory gets cleanup after the job, but sometimes it doesn't, which fills up my root directory. Is there a way to specify this directory so that it doesn't point to /tmp? I tried SPARK_LOCAL_DIRS, but it doesn't help, neither does SPARK_WORKER_DIR. Thanks. Justin
