Updated spark-defaults and spark-env: "Log directory /home/hduser/spark/spark-events does not exist." (Also, in the default /tmp/spark-events it also did not work)
On 30 March 2015 at 18:03, Marcelo Vanzin <van...@cloudera.com> wrote: > Are those config values in spark-defaults.conf? I don't think you can > use "~" there - IIRC it does not do any kind of variable expansion. > > On Mon, Mar 30, 2015 at 3:50 PM, Tom <thubregt...@gmail.com> wrote: > > I have set > > spark.eventLog.enabled true > > as I try to preserve log files. When I run, I get > > "Log directory /tmp/spark-events does not exist." > > I set > > spark.local.dir ~/spark > > spark.eventLog.dir ~/spark/spark-events > > and > > SPARK_LOCAL_DIRS=~/spark > > Now I get: > > "Log directory ~/spark/spark-events does not exist." > > I am running spark as "hduser", which I also use on the cmd(as verified > in > > the stdout "Set(hduser); users with modify permissions: Set(hduser)"). I > am > > able to cd into this directory. I can also create, view and delete files > in > > this directory, logged in as hduser. I checked the folder, it is owned by > > hduser. I even performed chmod 777, but Spark keeps on crashing when I > run > > with spark.eventLog.enabled. It works without. Any hints? > > > > Thanks, > > > > Tom > > > > > > > > -- > > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-events-does-not-exist-error-while-it-does-with-all-the-req-rights-tp22308.html > > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > > For additional commands, e-mail: user-h...@spark.apache.org > > > > > > -- > Marcelo >
