Anyone can help?

How can I configure a different spark.local.dir for each executor?


On 23 Mar, 2014, at 12:11 am, Tsai Li Ming <mailingl...@ltsai.com> wrote:

> Hi,
> 
> Each of my worker node has its own unique spark.local.dir.
> 
> However, when I run spark-shell, the shuffle writes are always written to 
> /tmp despite being set when the worker node is started.
> 
> By specifying the spark.local.dir for the driver program, it seems to 
> override the executor? Is there a way to properly define it in the worker 
> node?
> 
> Thanks!

Reply via email to