I'm running on EC2 and I want to set the directory to use on the slaves
(mounted EBS volumes).

I have set:
    spark.local.dir /vol3/my-spark-dir
in
   /root/spark/conf/spark-defaults.conf

and replicated to all nodes. I have verified that in the console the value
in the config corresponds. I have checked that these values are present in
nodes.

But it's still creating temp files in the wrong (default) place:

    /mnt2/spark

How do I get my slaves to pick up this value? How can I verify that they
have?

Thanks!

Joe

Reply via email to