I have set spark.eventLog.enabled true as I try to preserve log files. When I run, I get "Log directory /tmp/spark-events does not exist." I set spark.local.dir ~/spark spark.eventLog.dir ~/spark/spark-events and SPARK_LOCAL_DIRS=~/spark Now I get: "Log directory ~/spark/spark-events does not exist." I am running spark as "hduser", which I also use on the cmd(as verified in the stdout "Set(hduser); users with modify permissions: Set(hduser)"). I am able to cd into this directory. I can also create, view and delete files in this directory, logged in as hduser. I checked the folder, it is owned by hduser. I even performed chmod 777, but Spark keeps on crashing when I run with spark.eventLog.enabled. It works without. Any hints?
Thanks, Tom -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-events-does-not-exist-error-while-it-does-with-all-the-req-rights-tp22308.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
